If you get user complaints about broken CalDAV syncs with Horde, there’s many places to look at. In one particular instance, an event was created from travelling app Transportr into the stock android Calendar app. Through the DAVx5 sync app, the user wanted to push these events to Horde’s SabreDAV interface – and from there, also sync it to his desktop email solution, Mozilla Thunderbird.
5 programa efectivo de entrenamiento de culturismo – programa de entrenamiento de culturismo de 5 días estore de culturismo halotestin szkani hombre sin mangas con capucha chaleco de fitness culturismo largueros entrenamiento camisetas sin mangas.However, his sync application told him about an error. The server administrator saw a 500 status code in the server log.
1.1.2.11 - user [24/Jan/2021:18:48:26 +0000] "PUT /horde/rpc/calendars/user/calendar~KL14jYhCMpbet4ecYzAg1bn/2bace303-f0d8-4df6-9652-baa5fb9e86c4.ics HTTP/1.1" 500 892 "-" "DAVx5/3.3.8-ose (2021/01/13; dav4jvm; okhttp/4.9.0) Android/11
The root cause was actually not in the software code but in the MariaDB database configuration. The calendar entry from Transportr included some Unicode icon characters like a fast train and some arrows. These characters are part of the standard unicode encoding, utf-8.
Now you might wonder: New installations of mysql and MariaDB default to a character set they call utf-8 since 2010 or so. This shouldn’t be an issue. However, what they call utf-8 is not what you would expect.
Some years ago, TV sets which did not support the full HD resolution were marketed as “HD ready”. In some sense, the default character set is “unicode ready” at best. The default data type saves on disk space by encoding a subset of utf-8 into up to three bytes. While this supports most natural language characters, it is only a fraction of what Unicode can offer. Database manufacturers are well aware that this is not something you should run nowadays that unicode icons like the hamburger are all over the place is user generated content. The mysql manual even says:
“Please use utf8mb4 instead. Although utf8 is currently an alias for utf8mb3, at some point utf8 is expected to become a reference to utf8mb4. To avoid ambiguity about the meaning of utf8, consider specifying utf8mb4 explicitly for character set references instead of utf8.”
Now that’s what I did. First I changed horde’s database encoding to utf8mb4 in conf.php:
$conf['sql']['protocol'] = 'tcp';
$conf['sql']['charset'] = 'utf8mb4';
Then I changed the mysql server’s and client’s default charset:
cat /etc/mysql/my.cnf[server]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
[client]
default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4
[mysqld]
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
I have obviously stripped a lot of file content not relevant to our story. After reloading the database, all new connections should use the real utf8 encoding and new tables should be created with the new standard. But what about existing content? We need to convert all tables and all their text-like columns, varchars, mediumtexts etc.
First, it’s backup time – better safe than sorry.
Then, let’s find all tables in our db server and feed them conversion commands.
mysql --database=horde -B -N -e "SHOW TABLES" | awk '{print "ALTER TABLE", $1, "CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"}' | mysql --database=horde
Do this in a downtime or ensure HA some way or other. It will take some time. Some sources suggest it might be sensible to also rebuild the tables with the optimize command. I am not very convinced, but it won’t harm.mysqlcheck --auto-repair --optimize --user="hidden" -p --databases mysql horde
After this, repeat the sync test. It should work this time.