Making plans is great but it’s just words. How are things going? Clearly moving towards goals but in unexpected ways.
Trains don’t only run on railroad tracks but supposedly on schedule. Commuters shall know every stop, arrival and departure, the order of cars and their position on the platform. If trains regularly arrive delayed, at another platform, composed of different or differently ordered cars it is a source of frustration. If they skip a station or get relayed to another course, it’s even worse. Software projects, on the other hand, are expected to detour regularly in very similar ways but hopefully arrive at certain way points more or less on time. Sometimes they get unplanned additional cars along the way. Sometimes you have to replace the engine. It never gets boring.
What is the goal?
In January, I anticipated there were some unknowns ahead but I did not detail how to tackle this.
Improve JMRI‘s support for the Märklin MCAN protocol and particularly the Mobile Station 2, Central Station 3 and Can-Digital-Bahn products. This might become a bit controversial.
The Märklin Hardware Throttles (Mobile Station 2) implement hot plugging and negotiate which one acts as the primary. They send a regular PING command announcing their type and serial and expect a PONG response from other devices on the bus. The most senior serial number is agreed to be the primary throttle even if the other one was the primary or single throttle until now. The primary offers its roster to the other throttles. If higher end Central Station systems are on the bus, one of the central station will always be the primary device and the throttles all behave a little different. This allows JMRI to passively listen to traffic and detect bus devices or actively send a PING and collect the PONGs.
The Can-Digital-Bahn devices have a similar PING/PONG mechanism used by their configuration utilities. Unfortunately there are different configuration utilities for different generations of their sensors, relays, turnout controllers, light controllers, switchboard components, mixed devices. These utilities are closed source Windows programs and you need different versions of the program for different versions of the components. Unfortunately, not all of the programs properly run on recent Windows systems.
What if I just had to push a button in the web browser and see all my MCAN throttles and devices in a table, loading the right configuration screen for each of them? Sounds much better, but how to achieve that?
Moving through the stack
To send the PING messages and receive the PONG messages a program needs to connect to the MCAN bus. This can be achieved by connecting the CC-Schnitte interface to the USB port of the computer or connecting to a Central Station device over TCP/IP. Both methods expose the raw MCAN bus to the software. Both options are already implemented in JMRI.
Next the necessary bus messages and response need to be added to the software. The MCAN PING command is already implemented but JMRI doesn’t implement the desired reactions to the answers. For can-digitial-bahn messages, support is still missing.
To use the found devices, a program needs to memorize some representation of them. JMRI currently does not have a generic way of handling external bus devices as such. Throttles and sensors are tracked in their specific roles, covering their common aspects as throttles or as sensors. For CBUS type connections, there is a node manager which looks very similar to what is needed for MCAN but the UI code and the table format probably deviates a bit. This is where it becomes a bit complicated.
Users need some kind of GUI to interact with found devices. There are many reasons to start out with a browser based UI instead of building it in JMRI’s native Java GUI. The browser based ui just works on any tablet or laptop connected to the network. It’s well decoupled from JMRI’s core. It can be styled or completely recomposed as needed without touching JMRI itself. JMRI does not need to bundle variants for different use cases but they can be installed separately, developed on their own schedule. JMRI only needs to expose the necessary interface protocol which also drives other parts of the browser based UI.
JMRI’s interface protocol is composed from JSON messages between the browser client and the JMRI server. These can be sent over the HTTP server port or through a separate WebSocket implementation. The latter provides better performance and less overhead.
JMRI’s WebSockets and HTTPS API
This is what I ended up doing. To familiarize myself with the API I built a simple demo use case to manipulate the displayed railroad company name. Previously this was only possible through the preferences screen in the native Java GUI. I ended up not integrating it in the regular browser based GUI but implement a separate demo page. JMRI core developers pointed out that special care is needed and unauthenticated calls to the API must not actually persist the changed configuration into JMRI.
Obviously this would not work for actually handling hardware. While JMRI already has a permissions and user authentication system, it is fairly new and does not cover the actual API messages. So my next step would be to implement the necessary messages in the JSON WebSocket API for retrieving an authentication token. This token would then be used to authorize further WebSocket calls which actually change something in JMRI. These new calls need to carry the authorization token as part of the data.
I plan to detail this development in a separate article later this month as I move through the process.
Seems like I should be getting somewhere
This free time project is a great adventure into the unknown between where I am and where I want to be next. In some ways this is similar to professional work. I commit to goals and target dates and I have a very clear understanding of next week and some ideas about the week after. Beyond that the way points and schedule dates become rarer.
It’s sometimes very challenging to tell project managers that there won’t be many super detailed milestones beyond the horizon of immediate next steps. Gladly, at work I have very smart project managers who know when to trust me and when to challenge me. Sadly, in free time I have a very harsh and unforgiving project manager who often won’t take no for an answer.