I'm running my own XMPP chat server with the help of Prosody, and many of my family are using it.
A couple of months ago, I got XMPP spam: Users from foreign servers would appear in my chat program and ask to be added to my address book. Others simply sent a single message with an advertisement for a russian XMPP spam service.
The same thing started to happen again two weeks ago, but this time I was not the only target: One of my family members also got spammed. Spammers ruin everything.
The relative in question has no contacts on other servers and will most likely never get any, so I simply wanted to block all external messages to him: Enter mod_firewall.
At first nothing worked. I added a debug rule that should only log all incoming messages to a given user:
TO: user@example.org LOG=[info] User received: $(stanza) PASS.
I had no messages in my log file. The prosody support chatroom then gave me the hint I needed; the firewall_scripts line in prosody.cfg.lua was not placed in the "global" section but after a VirtualHost and thus was only loaded for that host.
Now I could finally write a firewall rule that blocks all incoming messages to a specific user:
ENTERING: $local TO: user@example.org NOT IN ROSTER? LOG=[info] Message to user blocked from: $(stanza.attr.from) BOUNCE=not-allowed
I finally found the time to make my errbot plugin errbot-exec compatible with errbot version 5 and Python 3, so I can finally use it again on Debian 9.
When starting errbot, I got a connection error:
$ /usr/local/bin/errbot --config=/etc/errbot/bookmarks.py ... 22:23:52 ERROR sleekxmpp.xmlstream.xmlst CERT: Invalid certificate trust chain. 22:23:52 ERROR sleekxmpp.xmlstream.xmlst Can not read from closed socket.
After enabling debug logging in the config file, I got more information:
22:23:53 DEBUG sleekxmpp.features.featur Starting TLS 22:23:53 INFO sleekxmpp.xmlstream.xmlst Negotiating TLS 22:23:53 INFO sleekxmpp.xmlstream.xmlst Using SSL version: TLSv1 22:23:53 ERROR sleekxmpp.xmlstream.xmlst CERT: Invalid certificate trust chain.
Then I used the XMPP observatory to get information about my own server's SSL configuration.. and saw:
TLSv1 No TLSv1.1 Yes TLSv1.2 Yes
I suspected that sleekxmpp tried to connect with TLS v1.0 and failed because of that.
After enabling TLSv1.0 in ejabberd.yml, errbot could connect without problems.
At work we're unfortunately using Slack for single and group chats. Most people use its web interface, some installed their web interface into a memory and CPU hogging electron desktop application.
I've been using the Psi XMPP client
because I also have other networks to connect to.
But now Slack told me We will close the gateways on May 15th, 2018
.
The bullshit about how the XMPP protocol isn't suited for their needs,
which is
a total lie.
The truth is that they did not care at all about their Slack gateway. I opened a number bug reports, and not a single one of them got fixed:
Joining Chatrooms with Psi+ XMPP client not possible
Support Request #583350, 2015-10-19
The point is that using an account's "service discovery" window, you can see all chatrooms - but you can't join them there.
Slack's suggested workaround was to use the "Join Group Chat" menu item and manually type all the chat room information to join.
XMPP: Automatically join chatrooms
Support Request #585019, 2015-10-20
It's currently not possible to automatically join chatrooms when connecting with an XMPP client to slack. This means I have to spend 2 minutes each day connecting to each chat room manually and having to remember all the chat room names.
This is because slack does not implement XEP-0048 (Bookmarks), which would allow my XMPP client to store bookmarks and the auto-join setting for a chatroom. http://xmpp.org/extensions/xep-0048.html
Their answer:
Unfortunately we do not support gateway bookmarking and currently have no plans to because not all XMPP clients support it.
XMPP: profile image very low-res
Support Request #594033, 2015-10-27
The profile images of users that get transferred via the XMPP gateway are of very low resolution. I can hardly recognize the people on the images in my XMPP client.
It was never fixed.
Show offline contacts as "offline" in XMPP, not as "away"
Support Request #584247, 2015-10-19
Slack people who are not logged on are shown as "away" in my XMPP client. This means they are still shown in the list of chat contacts.
I'd rather see them as "offline", so that they don't even show up in the list of contacts. If I wanted to see them, I can still "show offline contacts". But as it is, the "away" status of in-reality-logged-off-people is bothering me.
They responded and told me that they can't do that because on XMPP you can't message users that are away - which is a flat lie.
Then they said there are some XMPP clients that don't support the chat-to-offline-people feature, but didn't mention which clients this should be.
The issue was never fixed, my roster was basically unusable with 100+ people always being shown.
XMPP gateway: Profile empty
Support Request #583343, 2015-10-16
The XMPP profile is empty; I don't see my colleague's full names - only their nick names. The e-mail address field is also empty.
I debugged the issue, and found:
The XML when requesting the vCard contains the <FN> within the <PHOTO> XML tag, which is wrong. It should be outside the PHOTO tag.
They never fixed it.
XMPP: profile image very low-res #2
Support Request #1323705, 2017-05-02
I opened that because the original issue could not be commented on after 1.5 years of inactivity. I never got a response.
XMPP: Automatically join chatrooms #2
Support Request #1323707, 2017-05-02
I opened that because the original issue could not be commented on after 1.5 years of inactivity. I never got a response.
XMPP gateway: Profile empty #2
Support Request #1323704, 2017-05-02
I opened that because the original issue could not be commented on after 1.5 years of inactivity. I never got a response.
xmpp gateway: <photo> in presence all have the same image
Support Request #1377805, 2017-06-22
I'm using Psi+ version 1.0.9 to connect to the slack xmpp gateway. All people in the contact list have the same picture.
This is because the XMPP gateway sends the same photo for all people - see the attached XML file with some presence information. All have the same photo tag content:
<photo>sha1-hash-of-image_208073</photo>
They told me they can't reproduce the issue with other clients and did not even look at their XMPP XML that clearly shows the problem.
Of course it was not fixed.
Slack never wanted to have a good XMPP gateway. Now that they are big enough to fail, they can shove this decision down their user's throat and shut XMPP and IRC gateways down.
I'll do everything I can to get my workplace to switch away from Slack.
I'm storing bookmarks in a SemanticScuttle instance on my home server, to which I only have access via VPN. Since I don't want to run my bookmarking instance on a public server, bookmarking interesting things at work involved sending emails to myself and storing them later at home.
So my dream for years was to have a bot in my Jabber roster that I could paste URLs, and it would automatically bookmark them in SemanticScuttle.
Six years ago I even started to fix up the xmpphp library with the goal to some day write a bot with it. But it was never stable enough for me, and lacked MUC support. I abandoned it when that became obvious.
Fast forward to now, and the problem and its accompanying pain was still there. Unfortunately the XMPP client daemon idea never took foot on Linux (although the Pidgin developers had a ticket for it and even had a GSoC project), so I had to look for for XMPP bot implementations.
The most promising software I found was errbot. It's written in Python, has many commits, is developed actively and has nice plugin writing documentation .
I had looked for a bot which also provides an init.d script, but errbot unfortunately does not provide one . (This isn't a problem, more about that later.)
errbot is still Python and not PHP, and I did not want to write my SemanticScuttle bot in Python. My plan was to write an errbot plugin that executes a program for every message sent to the bot, and sends its output back to the user.
errbot has a plugin template that you can use to kickstart your own plugin. Simply run errbot --new-plugin and you're done.
For development, errbot has a local test mode that does not connect to any server, which means quick bootup times.
While writing the plugin I learned about os.exec* and os.popen, and that I should rather use the subprocess module for starting programs in python.
By default, errbot plugins are configured through chat commands . I did not like this approach and implemented a way to configure the executable directly in errbot's config file.
In the end, the plugin consists of two files with 88 lines of code and does what I want: Execute the new bookmark-bot.php script in SemanticScuttle's scripts/ directory.
I can bookmark URLs via chat:
I'm running errbot on my home server, which I had to update from Debian 7 to Debian 8 because errbot failed with Debian 7's python 2.7.3, Debian's python 2.7.9 works fine though.
After SemanticScuttle was update to the latest git version, errbot itself gets installed via pip:
$ apt-get install python-pip
$ pip install errbot
Then I cloned the exec plugin into /usr/local/src:
$ cd /usr/local/src
$ git clone git://git.cweiske.de/errbot-exec.git
After that, errbot needs to be configured. Since configuration files are stored in /etc/, I created an errbot dir and put the config file in there:
$ mkdir /etc/errbot
$ emacs /etc/errbot/bookmarks.py
The configuration file is as minimal as possible:
import logging
BACKEND = 'XMPP' # defaults to XMPP
BOT_DATA_DIR = '/var/lib/errbot/bookmarks'
BOT_EXTRA_PLUGIN_DIR = '/usr/local/src/errbot-exec/'
BOT_LOG_FILE = '/var/log/errbot-bookmarks.log'
CORE_PLUGINS = ('ACLs')
# DEBUG, INFO, WARN, ERROR.
BOT_LOG_LEVEL = logging.ERROR
BOT_ASYNC = True
BOT_IDENTITY = {
# XMPP (Jabber) mode
'username': 'dojo@cweiske.de/bookmarks',
'password': 'yesiwilltellyouthis',
}
BOT_ADMINS = ('gbin@localhost','cweiske@cweiske.de')
ACCESS_CONTROLS_DEFAULT = {
'allowusers': ('gbin@localhost','cweiske@cweiske.de'),
}
EXEC = {
'command': '/usr/local/src/semanticscuttle/scripts/bookmark-bot.php'
}
# unfortunately required settings
BOT_LOG_SENTRY = False
SENTRY_DSN = ''
SENTRY_LOGLEVEL = BOT_LOG_LEVEL
CHATROOM_PRESENCE = ()
CHATROOM_FN = 'Err'
BOT_PREFIX = '!'
DIVERT_TO_PRIVATE = ()
CHATROOM_RELAY = {}
REVERSE_CHATROOM_RELAY = {}
Data and log directory need of course be created:
$ mkdir /var/lib/errbot/bookmark
$ touch /var/log/errbot-bookmarks.log
errbot shall run with as less privileges as possible, so I created an own user for it:
$ useradd --no-create-home --no-user-group -g nogroup -s /bin/false errbot
$ chown -R errbot /var/lib/errbot/
$ chown errbot /var/log/errbot-bookmarks.log
Now I could already start it, but I wanted to run it whenever the server boots up. Debian 8 switched from SysV to systemd, and writing init scripts for it turned out to be easy.
Create /etc/systemd/system/errbot-bookmarks.service with the following contents:
[Unit]
Description=Jabber bot for bookmarking
After=network.target
[Service]
Type=forking
User=errbot
ExecStart=/usr/local/bin/errbot --daemon --config=/etc/errbot/bookmarks.py
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target
Run man systemd.service to see the full list of allowed directives.
Do not use a symlink! systemd behaves strange if the service script is not a real file.
Now we make it start at boot and start it ourselves:
$ systemctl daemon-reload
$ systemctl enable errbot-bookmarks
$ systemctl start errbot-bookmarks
$ systemctl status errbot-bookmarks
● errbot-bookmarks.service - Jabber bot for bookmarking Loaded: loaded (/etc/systemd/system/errbot-bookmarks.service; enabled) Active: active (running) since Fr 2016-05-13 15:21:10 CEST; 4 days ago Main PID: 30673 (errbot) CGroup: /system.slice/errbot-bookmarks.service └─30673 /usr/bin/python /usr/local/bin/errbot --daemon --config=/etc/errbot/bookmarks.py 2016-05-13 15:21:10,210 INFO errbot.err Config check passed... 2016-05-13 15:21:10,210 INFO errbot.err Selected backend 'XMPP'. 2016-05-13 15:21:10,211 INFO errbot.err Checking for '/var/lib/errbot/bookmarks'... 2016-05-13 15:21:10,211 INFO errbot.err Daemonizing Starting daemon.
Voila, the bot is in action!
At work we're unfortunately using Slack for group communication. I'm mostly communicating via Jabber and refused to install another chat client software - now I am utilizing the Slack XMPP interface.
Psi+ as chat client is working fine, but what annoyed me was my inability to see the emoticons my colleagues were sending. I only got :+1: and :simple_smile: instead of the smiley itself. It was time to change that.
Psi+ allows you to choose one or multiple sets of emoticons to use in Options -> Appearance -> Icons -> Emoticons. None of those supported the emojis used by Slack, so I had to create the set myself.
I found multiple image sources:
The emoji-cheat-sheet.com one had a github repository, which meant that I could access it easiest - so I chose it.
The Psi+ Ubuntu PPA contains the psi-plus-icons-nonfree package, which installs a number of .jisp files. Those files are application/zip files with a icon set definition file icondef.xml, and a number of .png image files that are referenced from icondef.xml. All those files are contained in a top-level folder that has the same basename as the .jisp file:
$ unzip -qql /usr/share/psi-plus/iconsets/emoticons/Emojis.jisp 0 2016-04-01 15:38 Emojis/ 1390 2016-04-01 15:34 Emojis/0023-20e3.png 1265 2016-04-01 15:35 Emojis/0030-20e3.png 1284 2016-04-01 15:35 Emojis/0031-20e3.png ... 1361 2016-04-01 15:36 Emojis/303d.png 1575 2016-04-01 15:37 Emojis/3297.png 1631 2016-04-01 15:37 Emojis/3299.png 90442 2016-04-01 15:39 Emojis/icondef.xml
The XML file itself is easy to understand:
Emojis
1.1
Unicode Emojis
2016-04-01
☺
:-$
:$
...
]]>
The incoming text of chat messages is scanned for the text values in the configured emoji icon sets and replaced by the image.
The icons are also shown when pressing the emoji button in the chat window, and clicking on one icon inserts the text that has the default attribute.
I did not want to write that icondef.xml myself. emoji-cheat-sheet's git repository contains a index.html file listing all images and the corresponding text, and I wanted to get XML from that.
What's the tool to convert XML to XML? XSLT .
index.html was HTML5 for no reason, so I had to pipe it through tidy (5.1.14 worked) to get clean XML. After that I could write my XSL script:
Emoji cheat sheet
0.1
from http://www.emoji-cheat-sheet.com/
2016-02-12
cweiske
: :
]]>
I had to convert the icons to 16x16, since Psi does not scale them itself; when I used the 128x128 ones I had a normal text line and huge smileys..
In the end I executed that commands:
$ git clone https://github.com/arvida/emoji-cheat-sheet.com ecs
$ tidy -f /dev/null -indent -asxml -o ecs-fixed.xml ecs/public/index.html
$ mkdir emoji-cs
$ cd ecs/public/graphics/emojis
$ for i in *.png; do convert "$i" -resize 16x16 "../../../../emoji-cs/$i"; done
$ cd ../../../../
$ xsltproc icondef-from-ecs.xsl ecs-fixed.xml > emoji-cs/icondef.xml
$ zip emoji-cs.jisp emoji-cs/*
Copy the generated .jisp to /usr/share/psi-plus/iconsets/emoticons/ and restart Psi+. Then open the appearance options and activate your icon set.
At work I recently wanted to let the central TV speak the company's main chatroom messages.
I'm using Psi+ to connect to our XMPP server. Psi writes chatroom logs into ~/.local/share/psi+/profiles/default/history/*.history, with one message per line.
At first I had to remove the timestamp from the messages, and then I had to filter out all non-alphanumeric characters which would not be spoken correctly anyway:
$ tail -n 1 -f .local/share/psi+/profiles/default/history/room\
| cut -b18-\
| tr -C -d '[:alnum:][:cntrl:] '
It did not work. Any single command after the tail worked, but piping several times did not.
After quite a lot of searching I found out that the pipe streams are buffered (cached). This is done to improve performance - with buffering, the tools receiving piped content will not have to handle each byte separately, but can do so in chunks.
Luckily there is a GNU coreutils command stdbuf which lets us modify the buffering settings:
$ tail -n 1 -f .local/share/psi+/profiles/default/history/room\
| stdbuf -i0 -oL tr -C -d '[:alnum:][:cntrl:] '\
| stdbuf -i0 -oL cut -b18-\
| spd-say -l de -e
stdbuf -i0 disables input buffering, and stdbuf -oL uses line-based buffering for the output, which suits my use case very well.
spd-say is the command line interface to speech-dispatcher.
Then I started pulseaudio-dlna to get a local audio output device that sends the data to the TV-attached Chromcast dongle, and used pavucontrol to set the speech-dispatcher audio output device.
This was the first time in my live that I liked pulseaudio.
We switched all our phones to Snom VoIP phones at work. They are managed by an Asterisk telephony switchboard server with the Gemeinschaft web interface.
One very cool thing about the Snom phones is that they are configurable in very many ways, and can fetch their configuration from the Asterisk server automatically. Once you find a setting that's useful for all, simply set it in the provisioning profile and let the phones reboot - and all have the new config setting.
One of the phone's features is calling an HTTP URL when something happens, e.g. start of an incoming or outgoing call, or connecting/disconnecting of the call. My self-assigned task now was to get chat notifications whenever a call comes in.
We use Prosody as XMPP (Jabber) server. The list of official plugins is long, and one of them is mod_post_msg . It allows you to send XMPP messages with a simple HTTP call.
Unfortunately, mod_post_msg only supported plain text messages - but I wanted to be able to link the names of the incoming caller to our internal address book, and also provide one-click redial/callback functionality via tel: URLs.
Prosody is written in Lua, a scripting language often used in games. With the help of the prosody MUC channel it was easy to extend the plugin with support for HTML messages. My patch got merged and can be used by everyone now.
When sending a HTML XMPP message, you should always provide a plain text message as fallback for clients that do not support, or whose users do not want to display HTML content. Apart from that, sending the message via a POST request is not hard:
'
. htmlspecialchars($callerName)
. ''
. ' '
. '☎';
$postMsg = 'to=' . urlencode($xmppTarget)
. '&type=chat'
. '&body=' . urlencode($textMsg)
. '&html=' . urlencode(
''
. $htmlMsg
. 'body>'
);
$ctx = stream_context_create(
array(
'http' => array(
'method' => 'POST',
'header' => array(
'Content-type: application/x-www-form-urlencoded',
'Authorization: Basic ' . base64_encode($xmppUser . ':' . $xmppPass),
),
'content' => $postMsg
)
)
);
file_get_contents('http://xmpp.example.org:5280/msg/', false, $ctx);
?>
]]>
The result will look similar to this:
At work, we use our own XMPP/Jabber server for corporate internal instant messaging and contact with permanent customers. Our jabberd2 server provides chat, group chat rooms and file transfers via proxy65. We have become very dependent on it, and cannot do and do not want to work without it anymore.
Every now and then, our internet connection breaks down and we are offline. Panic spreads then because we are unable to determine where to go for lunch, because chatting does not work.
A way to be able to chat even without having our XMPP server available is using link-local, serverless messaging as defined in XEP-174 . It uses zeroconf networking and the jabber protocol for direct client-to-client chat sessions. Some Jabber clients like Pidgin, iChat and Empathy already support that. All others are left out.
I personally like and use Psi for chatting, and Psi does not support serverless messaging. Swift, a new XMPP client contains Slimber, a link-local to xmpp proxy. Once started, it acts as both a local XMPP server your Jabber client can connect to and as a serverless XMPP client. With it, I am able to chat on the local network, even when our office is offline.
Installing it is easy from its git sources:
Now all that's left is adding a new account in your Jabber client. Choose a jabber id "username@localhost", disable SSL and allow plain text authentication on unencrypted connections. That's all.
At work, we use our own jabberd2 XMPP server instance to communicate with each other. Apart from one-to-one chats, we use the MUC plugin to get company-wide and project specific chat rooms.
The jabberd2 server directly reads user data from our mail server database (see XAMS patch, MySQL view for jabberd2 on the mail server DB). Our benefit in hosting our own jabber server is that we have absolute security by employing SSL connections that nobody can eavesdrop. Sensible data are only being sent over servers that we control.
One thing that did not work yet was file transfer. The idea is easy: Simply drop a file from your desktop of file manager onto the coworker in your XMPP roster (contact list) and the file is being sent to him. Sometimes it worked when both are in the internal network, but not always, and not with all clients. The solution here is either open ports in the company firewall or a file transfer proxy.
proxy65 is such a one; it is easy to setup and does what it says on the tin - even if the last release was nearly two years ago. Installation was easy:
After installation, you really should start proxy65 in debug mode:
twistd --nodaemon --logfile='-' proxy65 --secret=pass --proxyips=1.2.3.4 --rport=5347
When everything is fine and running, you should see a new XMPP service in the service list:
Now you may add that jabber ID ("proxy65" in our case") as file transfer proxy to your account settings. Please note that, although sometimes it looks like a FQDN like proxy.jabber.org, this is a Jabber ID! It took quite a while until I noticed that.