Movim 0.21, codename Whipple, is finally out!

Let's have a look of all the new and improved things that you can find in this big #release 🥳

Message replies

You can now reply to messages thanks to the implementation of the XEP-0461: Message Replies.

More and more clients in the XMPP ecosystem supports this feature, including Slidge, new XMPP gateways project that is allowing you to bridge Movim with Telegram, Discord and many others chat platforms.

Push Notifications

Movim now integrates WebPush. Never miss a message, even when Movim is closed. This feature is also working when you install Movim as a Progressive Web App on your Android or iOS device.

Improved account configuration

The configuration panel has been redesigned to be more accessible.

You can now block contacts directly from your Movim instance and manage your block-list from the panel.

Microphones and webcams can also be configured and tested from the Audio & Video configuration tab.

New emojis

This version brings the support of Unicode 14 and many new emojis that you can use in your messages, posts, replies and reactions.

Redesign

Movim is following the #Material Design guidelines since 2014. This release is bringing a fresh redesign of the components and animations based on Material 3.

The main menu was reorganized to clarify the navigation and hide the second-level pages in a sub-menu that appears when hovering the account item.

Following this redesign Movim accounts can now set a banner next to their avatars.

Share and Send To

The Send To feature, that allows you to send articles to contacts was completed by a Share feature allowing you to share the article in a new publication on your personnal blog on in a Community that you're in. Useful to share things around !

Audio messages

Movim can now play and record #audio messages.

Gallery Communities

When creating or configuring your Communities you can now set a Community type. The Gallery Community forces the publications to contain at least one image and display them as a grid.

This feature is the result of a long clarification and standardization work made on XMPP Pubsub with the pubsub#type attribute, the introduction of a new XEP based on that change called XEP-0472: Pubsub Social Feed and the support of pubsub#type in ejabberd (related ticket).

Performances, memory consumption and stability

A very important work was done to limit the Movim processes memory consumption.

The daemon and subprocesses are now using PHP Opcache to load and share only once lots of files that were previously loaded multiple time during the Movim runtime. PHP modules are also loaded using a predefined whitelist to limit the usage of useless modules in memory.

DotEnv configuration

The old configuration system has been moved to the DotEnv standard. This change merges all the previous configuration (database, daemon and paths) into an unique .env file.

They can also be set using environment variable directly in your Docker Compose, or Web Server (using SetEnvin Apache for example).

The official Movim Docker image was also updated to fit those changes.

Migration from Movim 0.20

If you are planning to upgrade your current Movim instance please follow those few steps:

1. Copy and rename the .env.example file in .env and complete the few configuration variables in it. They should be the same as the one you set in the previous db.inc.php file and your daemon parameters.
2. Remove the db.inc.php file
3. Remove all the daemon.php parameters from your init.d, systemd services or other scripts. The daemon launch command should look this way: $ php daemon.php start.

...and as always, don't forget to run the migrations (php composer.phar movim:migrate) and restart your daemon.

XMPP Pubsub node security and restrictions

Movim 0.21 is not trusting anymore posts, likes and comments that are not containing the explicit identifier (Jabber ID) of the publisher and therefore now rely on this part of the XEP-0060: Pubblish-Subscribe - 12.16 Associating Events and Payloads with the Generating Entity.

All the existing likes and comments might be not counted anymore or seen as "Non trusted" ones. All the new published ones will be configured properly.

Migration from Movim 0.20

On ejabberd

You can update all the existing stored node configuration to force the new default configuration using the following SQL request. You might do a backup of your database before doing such changes:

update pubsub_node_option set val = `publisher` where name = `itemreply` and val = `none`

...and load those changes without restarting ejabberd:

$ ejabberdctl clear-cache

On Prosody

Ensure that you have the expose_publisher = true set in your configuration, see the related documentation.

What's next?

Movim 0.22 should include two big projects.

OMEMO rewrite ?

The cleanup, rewrite and refactoring of the OMEMO support that is quite buggy and not opmized. We are not promising anything on this side, OMEMO is always a complex beast to handle.

Multi-part audio and video-conference feature

The audio and video conferencing features were already introduced a few years ago in Movim. Some preparation work has been done in this 0.21 release to be able merge back the pop-up video-conference window inside the main window for the upcoming release.

The multi-part audio and video-conference feature is also one of the main feature that miss in Movim and is requested quite often by our users. Let's see if Movim 0.22 finally include this long awaited #feature.

Regarding the amount of work that need to be done regarding those features it might be possible that specific funding will be requested for it to free up enough time to work properly on their integration.

Enjoy!

A big thanks to the #Movim community that is growing more than ever. You can find us on our main support chatroom movim@conference.movim.eu.

If you find issues or if you want to contribute to the project you can find everything on our Github page.

And if you want to support us, fund the development of new features and help us pay our servers, domains and communication we are actively looking for supporters and sponsors on our Patreon.

That's all folks!

Movim is joining the Mastodon network you can follow us on @movim@piaille.fr to stay informed of our latest news or you can continue to follow our official blog as well 😋

We also totally refreshed our Patreon page and introduced two new membership levels, Supporter and Sponsor. If you want to help Movim and fund our development do not hesitate to support us there.

We would really like to first cover our monthly expenses (servers, domains…) and if we get enough support be able to fund some new features such as group-videoconferencing 🥰

#Patreon #Mastodon #Sponsor #Support #Movim

Bye bye Facebook, bye bye Twitter, the federated platforms are ready to take over!

The whole Fediverse is booming, Mastodon looks like a really promising replacement for the little blue bird and Peertube to replace Youtube. Many other platforms are currently being developed around the ActivityPub ecosystem like explained in this article The Fediverse is so much bigger than Mastodon.

We think that Movim also fit perfectly in there by being a perfect blogging and chatting platform. Fully built on the widely used Internet standard XMPP it is packed with plenty of exciting features in a nice and friendly user interface.

The Libervia project is actually working on a bridge between ActivityPub (the Fediverse core protocol) and XMPP which would allow us to connect all those exciting platforms with Movim!

Feel free to share the word to help us!

We are just at the beginning of this exciting journey :)

edhelas

#movim #xmpp #activitypub #fediverse #mastodon #libervia #twitter #facebook

I was used to #release a new version of #Movim twice a year. Skiff is an exception. One year of work was required to release the 20th major version of the project.

The main reason is mostly based on the amount of work and adjustments required to integrate the main feature of this release: the support of end-to-end #encryption through the implementation of OMEMO.

So let's dive in all the new exciting features that you will discover in this major release.

OMEMO

The technical part was already extensively covered by the dedicated article End to end encryption in Movim - OMEMO is (finally) there!.

The user experience and flow is not very different than on other XMPP clients, if Movim detects that you can start an encrypted conversation with a contact a small lock icon will be displayed next to the chatbox. You can always choose to toggle it back to have a non-encrypted discussion.

It is also possible to see all the encryption fingerprints in the Contact drawer under the dedicated "Fingerprints" tab. You can also enable and disable encryption to each fingerprint manually there. Movim is displaying the last message sent or received and the client linked to the fingerprint to help you with your configuration. But rest assured, those settings are only for those that wants to configure in detail their encryption levels.

End-to-end encryption is also available for group chats, the flow is exactly the same as for single contacts.

There is some chances that you encounter encryption issues in some cases, even after a lot of debug and refactoring end-to-end encryption is a really complex beast that is difficult to handle. Feel free to open a ticket with all the details to reproduce the issue if you encounter one.

I'd like to thank again NLNet for their help on this project ! With the funding I was able to free-up time to finally integrate end-to-end encryption in Movim.

Posts

A few changes were made regarding the posts and their integration within Movim.

The post publication form was slightly redesigned and now allows several images, files or links to be attached. Linked to that change, post cards were also redesigned with a more compact design.

The public Communities and Blog pages now have the same 2-columns design as their private version. The displayed Communities and Contacts information are also now more compact.

The tags were redesigned and are now more clearly visible and navigable.

Chat

The contacts and chatrooms drawers were redesigned and now include some really useful information. Pictures and links sent in conversations are now quickly available in dedicated tabs.

Chat bubbles are now properly displaying quotes and support message styling.

A big refactoring was also done regarding how the edited messages are handled in Movim. This refactoring allowed messages to be edited in Group Chats and the support of several edits on a single message (which caused some weird message duplication bug).

Chatrooms

Chatrooms administrators can now manage affiliations and ban/unban users.

You can now prioritize your most important chatrooms on top of the list with the pin feature.

...and many other things

The old Movim API code was fully removed. It had been left untouched for years and not really used nor up-to-date anymore.

When you are in a chat conversation, the other chats counter is displayed on the back button.

The internal picture library was rewritten and simplified, it now supports transparent avatars. All pictures are now compressed in WebP by default.

Admins can now fully disable the registration feature. It is quite useful if you have a dedicated Movim setup and a specific separated flow to register your users (using an internal LDAP in a company or school for example).

Plenty of new emojis were integrated with the support of Unicode 13.0.

Movim is now a Progressive Web App

Movim used to have some "native" apps, on desktop and Android. All those app are now deprecated and replaced by work that was done to make Movim a full Progressive Web App. From any browsers you can now install Movim as an app on your phone or desktop in a single click.

Conclusion

Lots of other small improvements and features were integrated in this release but not listed there, it's time for you to discover them. Enjoy this new version!

That's all folks!

Once in a while I download and synchronize the #Movim #translations from the Transiflex platform.

Thanks to the awesome community, Movim is now translated in 57 different languages.

I was really surprised that the project was fully translated in Basque. A language spoken in the beautiful #Basque Country that sits between France and Spain. I personnally lived there for a few years (in Bayonna) and really loved those beautiful lands!

I'd like to thank again all the people that are working hard on the Movim translations!

Mila esker (Thank you in Basque) !

A few days ago I finally closed the OMEMO encryption ticket on Github. Opened in 2015 it had many twists and turns along the years but I finally found a proper way of integrating it in Movim.

In this article I'll explain why adding #E2EE (End to End Encryption) was not as easy as with the other #XMPP clients (and more generaly all the chat clients that are using a similar encryption protocol) and how I addressed the issue.

But before going in the details I'd like to thanks the NLNet Fundation for its financial support in this project. With their help I was able to free-up some time to work on the problem and propose a proper architecture (detailled bellow) for it.

The result of this work will be released with the upcoming 0.20 version of #Movim. There is still some quirks and whims about it but the base is there and works pretty well.

End to End encryption in XMPP, a quick overview

The introduction of Signal in 2015 brought a small revolution into the encryption protocols in the IM ecosystem. The Double Ratchet Algorythm (see the dedicated technical documentation on the Signal website) allowed users to exchange messages between different clients in an “end to end encrypted” way (only user devices themselves know how to encrypt and decrypt messages) with some technical improvements (not detailed here) that made the new protocol a “must have” for all the others IM solutions.

Today the Double Ratchet Algorithm is used in applications such as WhatsApp or Matrix.

In the XMPP ecosystem it was primarily pushed by Daniel Gultsch in the Conversations.im client and standardized along the way in the OMEMO XMPP Extension XEP-0384: OMEMO Encryption. Throughout the years many XMPP clients implemented OMEMO, their status can be tracked on the following website Are we OMEMO yet?.

The OMEMO architecture

Without going too deep into the technical details the general idea about OMEMO is to generate some keys on each of the user's devices and publish the public ones on their account server.

Using the keys published on the XMPP user's servers, anyone can then start an encrypted session at any time (the servers are always available) and start to send messages to the desired contact without having to wait.

If one of the user's contacts wants to start an encrypted discussion they will first start to get those keys, then build sessions with their secret one and encrypt a message using the freshly built sessions.

If a user receives a new encrypted message and doesn't have an encryption session to that device, their device will then retrieve the contact keys, build the encryption sessions and start decrypting messages.

This can be done automatically if the contact trusts blindly the key used or in a more “trusted” way by accepting manually each keys to build the encryption sessions on.

All the existing XMPP clients are using this simple architecture. XMPP servers are storing their users' #OMEMO public keys and the users are connecting directly using their different devices to build their encrypted sessions.

The Movim particularity

But Movim is kind of special. The XMPP connection is actually not maintained on user devices but by the Movim server (built in PHP and running behind a web server such as Apache or nginx, see Movim General architecture on the Wiki). Movim is then processing everything server side, saving the information (articles, contacts and messages) in a SQL Database (PostgreSQL or MySQL) and then showing the result to the Movim users through a lightweight website.

If a user is connecting on the same Movim instance through several browsers using the same XMPP account all the browsers are then “merged” into one unique XMPP session (called "resource") and all the browsers are synchronized in real time by the Movim server. This is pretty useful to save memory and to prevent Movim to maintain several XMPP connections at the same time for a unique user. This also allows quick disconnection/reconnections, the users can close and reopen their tabs without having to reload the whole XMPP state when they come back after a while (Movim is closing the XMPP session after a day of inactivity).

End to end encryption actually requires to encrypt and decrypt messages on the user device, this brings several issues:

• For Movim, the user device is actually a “dumb” browser that only display the messages pre-processed by the Movim server, there is no logic whatsoever browser side
• A user can use simultaneously several browsers with the same XMPP connection on the same Movim instance
• All the message processing logic is done server side

This unique architecture requires a very unique way of adressing the E2EE situation. Hopefully OMEMO offers all the tools needed to handle those cases.

Split the logic

The OMEMO extension is actually talking about devices, for a large majority of the XMPP clients a device is connected through a unique XMPP session (one device equal one current XMPP resource in those cases).

The fact that Movim is sharing a unique session (resource) with several devices (browsers) is actually not an issue in the end. Each browser will then be considered as a unique device on its own, with its own key and its own OMEMO encrypted sessions.

This brings some interesing results. When a user is connected using the same XMPP account using two different browsers on the same Movim server (also called instance, or pod), an encrypted message sent by the browser Firefox will then directly be decrypted by the browser Chrome without even having to travel through the XMPP network.

The term “browser” is also defining more than actual browsers (like Firefox, Chrome or Opera). Since we can have private navigation or containers (in Firefox) each time it is seen as a different “browser” on the Movim side (because each context is separated, with a different cookie and different local data).

So the global idea is to continue to handle the messages server side, push the encrypted message object to the browser, and then implement only the key handling and message encryption-decryption flow browser side. When doing this implementation I actually looked at the Converse.js and JSXC OMEMO implementations, the Movim implementation is really close to the one done on those two clients (I am also re-using the libsignal JavaScript implementation).

This architecture actually works for the current version of OMEMO (0.3.0) where only the body is encrypted. The upcoming versions are looking to encrypt a larger part of the XML stanza. This will be way more difficult to handle for Movim, as it will require to decrypt messages browser side and then implement a second parser, this time in JavaScript (everything is parsed in PHP using libxml at the moment).

if (textarea.dataset.encryptedstate == 'yes') {
    // Try to encrypt the message
    let omemo = ChatOmemo.encrypt(jid, text, Boolean(textarea.dataset.muc));
    if (omemo) {
        omemo.then(omemoheader => {
            ...
            xhr = Chat_ajaxHttpDaemonSendMessage(jid, tempId, muc, null, replyMid, mucReceipts, omemoheader);
            ...
        });
    }
} else {
    xhr = Chat_ajaxHttpDaemonSendMessage(jid, text, muc, null, replyMid, mucReceipts);
    ...
}

This little JavaScript Movim code extract presents the differences in handling encrypted and unencrypted messages. The text variable is containing the clear text version of the message. When the body is encrypted it is then calling the same method as for a clear text message.

This method is actually a wrapper generated by the RPC (Remote Procedure Call) Movim server core. Once this function is called an Ajax called is made and the rest of the flow is handled server side. The encrypted body, and generated OMEMO headers passed will be injected in a freshly generated XMPP XML <message>.

Keep the messages in the local database

With the separation of the logic it was then required to keep a copy of the decrypted messages browser side.

To do that an IndexedDB database is used. This database is quite simple and only contains a key-value store, where the key is the message id (the same as the one in the Movim server SQL server databse) and the value the plaintext message.

• When a message is decrypted the plaintext body is then stored in this database.
• When the user sends an encrypted message, the original text is also saved in this database.
• If a message cannot be decrypted, the message key is still saved in the browser database with a false value. This prevents Movim to try to decrypt several times a message, knowing that the decryption will fail each time in the end.

Using this database, when a chat is loaded, all the messages are then sent chronologically from the server, passed trough a little bit of code that will lookup the state of all messages and then decrypt the ones that are not decrypted yet, the already decrypted messages are then shown, or an error is displayed for those that cannot be decrypted.

To sum up

In this article I tried to present you what limitations I faced when trying to implement end to end encryption within Movim and what architectural and technical solutions were used to address them.

The current solution seems to fit and bring all the desired features to Movim without too much downsides. The feature can now be considered as done and will be released soon. And as always, lots of small fixes and adjustments will be integrated to polish it afterward.

That's all folks!

edhelas

It's been a while since I have told you about Movim. I was quite busy the past few months with many other things in my personal life, but this doesn't mean that I haven't worked on Movim!

So in short, here is a few things that have been changed recently:

api.movim.eu

api.movim.eu has been upgraded and refreshed with a new Laravel backend, Google ReCAPTCHA has been replaced with hCaptcha

The Legals page will be soon upgraded. A specific ticket has been opened to allow you to report and bring ideas on how to improve it.

End of the Android application

Following our previous blog post about the state of the Android application I decided to officially archive the Android app and remove it from F-Droid.

Some work has been done to integrate Movim better as a #PWA (Progressive Web App) to still allow you to enjoy it from your mobile phone.

Security Audit

We had an extensive #SecurityAudit by Radically Open Security that did an amazing job.

The audit covered Movim itself but also api.movim.eu and all the related projects, including movim/movim_docker. The result of the audit is currently restricted but we are actively using the finding to improve the general project security:

• There is now a SECURITY.md
• A few tickets were already addressed, on our infrastructure and the Movim code

An finally on Movim itself

• I added a basic support of OMEMO for MUC Groups, you can already try it out on the official pod
• And as always, bugs fixes!

That's all folks!

edhelas