Update: Operator Middleware
Hey folks, hope all is well out there. Today I want to share some info on Mugsy’s Operator Middleware.
Operator
First things first, this is not yet publicly available but will be by the end of this upcoming week. I will go over the remaining release todo items after the overview.
What is it?
Straight from the repo’s README:
This repository contains the source code for the Operator App, a FastAPI application designed to manage and control Mugsy’s hardware devices. All GPIO control is handled by Operator to keep hardware related functionality isolated from the primary DECAF api, allowing for increased hardware compatibility.
Requests from the Mugsy frontend are sent to the DECAF API which handles all primary functionality. When a GPIO action is required, DECAF calls Operator to perform the action.
Operator is the lower level control system for Mugsy’s GPIO related sensors and hardware. The DECAF API handles things like recipes, users, integrations, etc, while Operator has one job, control and management of GPIO pin state and connected hardware.
Keeping these two APIs separate from each other has a few significant benefits:
The frontend interface and DECAF do not need to know anything about the actual hardware the system is running on. This allows for the software to be migrated to lots of different hardware with minimal effort. It also will help manage compatibility between different Pi versions. Currently the middleware is only compatible Pi 3B+ and Pi 4. Version 5 of the Pi has some changes in how GPIO is accessed, so Operator will require a compatibility layer to match this updated setup. I don’t actually have a Pi 5 yet so Im not sure of all the details. This will be a straightforward process thanks to Operator’s design and will be an early item on the post release Operator roadmap. When a new compatibility layer is created, setting a flag for the hardware version in the hardware_config.ini is all that’s required to get the software running.
With Operator handling the hardware, the complexity of the rest of the systems goes way down. When updating the DECAF api with new features, there is no need to deal with any actual hardware control. A piece of hardware will only have a limited number of options, and those options are configured for the endpoint using simple Pydantic models. Those models can then be used as the basis for types in your new DECAF functionality.
Upon release, Operator can be used for any Pi based hardware projects, not just Mugsy. If you need a dead simple API for basic direct hardware control, Operator will likely be able to handle it. In addition to the built in configs for Mugsy’s brew related hardware and sensors, there are endpoints that are function agnostic and simply allow direct control and reading of GPIO.
Please note that the pour over mech and spout’s movement are not controlled by Operator. That control software is running on the microcontroller.
Features:
Easy configuration: To keep the package light and minimize resources, Operator requires no database for hardware details. All info about the available hardware is handled with the hardware_config.ini file. Here is an example showing the config details for Mugsy’s pump controller and relay board:
You’ll notice that there’s no config info about brewing coffee in there because Operator doesn’t care about that ( comments don’t count! ). It only needs to know where the hardware is and what its capabilities are. It’s up to DECAF to know what a recipe needs to get the desired result.
A DB is used when Auth is running on the endpoints. More details in the Auth section below.
Watchtower: Watchtower is a service that starts before the Operator service and keeps an eye on GPIO states. When an endpoint performs an action on a pin, it also sends those details to Watchtower over a socket connection. Here is an example:
The relay endpoint receives a POST command to turn on relay channel 1 for 5 seconds.
Before the pin state is updated, it sends the pin number and the time delay to Watchtower.
Relay state is updated.
Watchtower starts its own timer. If for some reason the API did not turn off the relay in the expected time frame, Watchtower will intercede and shut off the pin itself.
There is also a config setting for a default Watchtower timer. If you set it to 10 minutes, it will check the status of all relay pins and turn them off if they have been on longer than 10 minutes, even if they are in the expected state according to the last request.
In flight terminal logging while testing Watchtower.
Pump service: The pump endpoints provide the following functionality:
Run pumps at full speed for x seconds.
Run pumps at a specific flow rate for x seconds.
Run pumps at full speed or specific flow rate by weight. This functionality reads the weight service values for the mug platform scale sensors continuously while pumping to measure the exact amount of water requested.
Flow rates are handled by sending a PWM signal to the DRV8871 motor driver that handles the two water pumps. For Mugsy’s default setup, anything lower than 40% of max speed is basically doing nothing. This value can be set in the hardware config under MIN_FLOW_RATE. The endpoint is smart enough to reply with an error if a request is sent with a flow rate lower than this value.
Weight service: This service has two main functions, to calibrate the scale as needed and to continually track the weight sensors in the mug platform and the cone mech scale. Other services connect to it when performing a set action that requires a process to be run until a specific weight has been reached.
Relay service: The relay endpoints provide the following functionality:
Turn relay channel on
Turn relay channel off
Turn on relay channel for x seconds and then turn it off.
Turn on relay until x weight has been reached. This is how we handle grind weight.
Turn all channels off.
RFID Service: This service runs continually and sends an outbound request to the DECAF API with the tag’s UUID anytime a tag is read. Decaf will then find the user action linked to that UUID and perform it. By default, this action is set to brew your ‘Coffee Now’' recipe when it reads you mug’s RFID sticker.
Pin service: These endpoints allow similar control to our relay, weight and pump services, but with out the constraints of Mugsy specific hardware. So you can read sensors, control other hardware, etc, just by sending a pin number, an input or output state and a related action. You can also send options that will result in actions like ‘turn pin high until x occurs” to perform actions that require multiple components or functions. These endpoints can be used as the foundation for additional functionality or functionality completely unrelated to Mugsy.
Auth service: As noted in the configuration section, Operator does not require a DB for hardware details. But it does use one if you have endpoint Authentication enabled, which is the case in Mugsy’s default configuration. This is also the case for the DECAF API, Auth is always enabled by default.
This service receives a bearer token in the API request and confirms that the request is legitimate. Auth can be disabled with a simple flag for those who are customizing their setup or using Operator for other projects. Those options are out of scope for this overview.
The DB instance used for operator authentication is a single table and uses minimal resources.
Testing: All endpoints and services have robust tests, implemented in Pytest. Test coverage is currently sitting at 91%. All GPIO functionality is mocked in tests, so coverage will never actually get to 100%. Tests also run using Github actions when merging PRs to the dev and main branches. Failing tests prevent merges.
Remaining Todo Items for Release
Move config options for some endpoints to hardware_config.ini
Move some functionality out of the routes and into the endpoints service modules.
Updates to setup.py to install all related full time services.
Update the Operator service to not start if Watchtower is not running. This is already checked per endpoint, but I want to have Operator rely on it as a whole.
These are all simple tasks and Operator will be released within the next week.
I have found that jumping around at this stage is killing my ability to complete big ticket items, so from here on out I will only being working on a single bucket until it is done. I will work solely on Operator until its release. When Operator is released, I will move back to Cloud Control until it is ready for users. Brew testing will continue as testing these services requires it.
Also, related to brew testing, I recently discovered these silicone travel V60 cones that are 100% compatible with Mugsy’s cone mech. It’s so perfect that it feels like it was designed for it:
The results are quite surprising, really nice brew… I don’t have much else to say about it right now, but I feel like its going to come up again soon, so stay tuned.
OK, that is all for today. Next update will be when Operator is live, no later than the end of this week. I will also do a run down on remaining items for project completion and the estimated scheduling for those items.
Cheers!