Last modified: 2021-10-21 20:58

otg-disk - On-the-Go File Server

I have a Raspberry Zero (without wireless) equipped with sensors to capture environmental data. It's build to be deployed somewhere, data collection is started with a button. Same for shutting down the device. To get the data I connect it with OTG USB to another computer. I thought it would be nice if the Raspberry would automatically start a file server to let me pick the data ... and shutdown the device when I'm done.

What otg-disk does is

Effectively you get a simple administrative interface to the computer. The USB restriction can be bypassed (to run a local hotspot for file exchange) and the script can be installed as a systemd service unit.

Quick check

This is ia quick check for impatient readers who want to check first what the script is doing and if it's worth reading more.

  1. Make sure you have smbd and inotify-tools installed and the smbd service stopped.
  2. Load the script into an editor and set the variable WLAN_COUNTRY_CODE to match your country code (e.g. DE for Germany).
  3. Run the script from the terminal: ./otg-disk _force_.
  4. Connect to your host with your desktop's file manager by going to smb://your-host/ and navigate to the sys directory in the disk share.

In case you don't have your Raspi ready or not configured as OTG gadget: this should also work on any other Linux computer. (But there it doesn't make the fun to have that service.)

Configuration

The otg-disk script is self-contained: It contains all scripts and configuration that is needed to run it. Of course you have to provide the required binaries, see the next section.

The script configuration is stored in environment variables at the top of the script.

# DIR is our temporary directory.
DIR=/run/$NAME

This defines a temporary directory for the script's own use like storing pidfiles. This is a changeable option but you shouldn't touch that. /run/$NAME evaluates to /run/otg-disk which will not conflict with other applications and the directory lives on a temporary mount and is empty on every boot.

# We run as this user.
USER=pi

This is another don't-touch option, the script's username. To interact read-write with the samba server this must match the user under which you are normally working.

The following options are more interesting.

# What is the directory of the public samba share?
PUBLIC_DISK="/home/$USER/tmp"

This defines a directory that the samba server exports read-write to all users that can reach the Raspberry. And even better: the samba server will not ask for a password. So this is really a public location. When you are using otg-disk in a purely OTG context this doesn't matter because then access is limited to the USB host. When you activate otg-disk with wireless on the share is still protected by the W-LAN password. But you don't want to connect such a thing to the Internet.

# This is the location of the control directory.
SYS="$PUBLIC_DISK/sys"

This will be the location where otg-disk creates and monitors file changes to start and stop services or shutdown the system.

# Set these permission on the share directories to make smbd
# allow writing files.
DIR_PERM=775

Without permission 775 (aka rwxrwxr-x) smbd will not allow write access on the directories $PUBLIC_DISK or $SYS. I'm pretty sure that it was working with 755 when I started with the script and it changed after an apt upgrade. So it might be simply a minor smbd configuration change and therefore I made the permissions a variable.

# The SYS directory gives control to these services.
SVC_LIST="vnc httpd"

otg-disk can start and stop services based on changes in the samba share. This variable defines the service list. But of course, you need to add support for every service you put on the list on your own. At least for now there is no generic addition of services (like a directory containing scripts or unit files). Also the http server used by otg-disk is mini_httpd, which needs to be installed if you want to use that.

The following two parameters configure mini_httpd.

# Parameters for mini_httpd.
HTTP_PORT=2080
HTTP_DIR=$PUBLIC_DISK

They set the server's TCP/IP port and document root.

These variable are for the Raspi's hotspot - if it comes with a W-LAN chip.

# Hotspot configuration.
WLAN_COUNTRY_CODE=XX
WLAN_SSID=MyHotspot
WLAN_PASSWD=MySecretPassword

Make sure to set the correct country code for your location (e.g. DE for Germany) otherwise your hotspot may be illegal (and not properly working). Set SSID and password as you like.

The access point is one thing and the client IP-configuration is another.

# Access point network.
AP_IFC=wlan0
AP_IP_ADDR=192.168.1.1
AP_NETMASK=255.255.255.0
AP_DHCP_DEF=192.168.1.11,192.168.1.55,255.255.255.0,24h

These four parameters configure the Raspi's hotspot network (192.168.1.1/24) and the DHCP server in dnsmasq (DHCP range is from 192.168.1.11 to 192.168.1.55). AP_IFC needs to be changed if you have multiple W-LAN adapters and the hotspot is not running on wlan0.

The following variables are for smbd and shouldn't be changed. The logfile goes to the temporary directory we defined at the beginning. Its maximum size is 100 kBytes (samba configuration) and it's only meant to debug current but not past problems.

# Location of dynamic smb.conf and smbd's logfile.
SMB_CONF="$DIR/smb.conf"
SMB_LOG="$DIR/smb.log"

smbd's configuration files lives also in the temporary directory. It is created every time otg-disk starts. If you want to work with a permanent file and individual configuration notice that otg-disk deletes an existing $SMB_CONF and you have to disable that.

# Debugging issues with the script?  Setting this to `yes` disables
# the systemd service on startup.
OTG_FAILSAFE=no

otg-disk will reboot or shutdown the Raspberry whenever it sees changes on some particular files. And since the script is started at boot time it may send the computer into an infinite reboot loop if really all things fail really hard. If that happens and you want to debug it or if you want to try the script as service first then set this variable to yes. otg-disk will then disable its service when it starts. (It will also provide the file reenable-service in its $SYS directory to simplify starting script as service during the next boot.)

Finally,

# sys-monitor writes some log messages.
export LOG=yes

makes the $SYS monitor script write some log messages. If otg-disk is started from systemd the messages will go to the system logs.

The file server smbd uses it's own configuration file and this is included as so called shell "here document". (Search for the string global and there it is.) If the file server is running you will find the file as /run/otg-disk/smb.conf. Here is its contents:

[global]
  workgroup = WORKGROUP
  # 100 kBytes of log in temporary space
  log file = /run/otg-disk/smb.log
  max log size = 100
  logging = file
  log level = 0

  panic action = /usr/share/samba/panic-action %d
  pid directory = /run/otg-disk

  server role = standalone server
  map to guest = bad password
  guest account = pi

# This make a public writeable share.
[disk]
  browseable = yes
  path = /home/pi/tmp
  guest ok = yes
  read only = no
  create mask = 777

You can edit it there for debugging puposes (this smb.conf is not permanent) and reload the configuration with pkill -HUP smbd.

Installing binaries

otg-disk brings all scripts and configuration to run a public ad-hoc file server. However, the samba server needs to be installed.

The commands

sudo apt update  &&  sudo apt install samba inotify-tools
sudo systemctl stop smbd
sudo systemctl stop nmbd
sudo systemctl disable nmbd snmd

download the service and disable their system services. In case you are already using smbd on the Raspi notice that otg-disk does not stop (and restart) an already running smbd. (This may be added in a later version.)

The inotify-tools are required to monitor the $SYS directory for changes.

otg-disk supports mini_httpd as web server. But this is just for demonstration purposes. If it is not installed (because you don't need it or use something else) otg-disk will work without it. In case you want mini_httpd you can install it with

sudo apt update  &&  sudo apt install mini-httpd
sudo systemctl stop mini-httpd
sudo systemctl disable mini-httpd

(Notice that the package's name uses a - instead of _.)

I think (but I'm not sure) that the programs hostapd and dnsmasq, which are required for the hotspot come already with the OS installation.

Adding services

You may add your own services to otg-disk but you have to add the start-stop logic to the script. Services are started and stopped from within a long case statement:

case "$cmd" in
vnc)
    # First, terminate a running instanace, then start a
    # server, if required.
    vncserver-virtual -kill :1
    [ "$op" = "start" ]  &&  vncserver-virtual :1
    ;;

The variable $cmd is the service's name (the service's filename must match the case selector) and $op is either start or stop which means that the service should be started or stopped. The code snippet shows the basic logic:

With this simple approach a service is never run twice which could create issues.

In the example above vncserver does two things:

  1. It goes into the background allowing the script to continue while it is running, and
  2. provides a simple method (the -kill option) to stop its service.

Not all services work like that.

httpd)
    # The httpd does not come with a kill operation as vnc.
    # Therefore we keep its process id and terminate it by
    # that.
    kill_process "$DIR" httpd mini_httpd
    if [ "$op" = "start" ]; then
        mini_httpd -D -p $HTTP_PORT -dd $HTTP_DIR -l /dev/null &
        echo $! >$DIR/httpd.pid
    fi
    ;;

I'm a little bit cheating here. mini_httpd can go into the background on its own (not requiring the & at the end of the command line) and write a pidfile with the -i option, see man mini_httpd.

Here again the service is terminated first. mini_httpd doesn't do that on its own, so the shell function kill_process (included in otg-disk) is called: it stops the service with the process id that is stored in /run/otg-disk/httpd.pid if the process' name is mini_httpd.

If the operation is start mini_httpd is started and put into the background by the shell (to demonstrate how this works; mini_httpd is artificially kept from doing this with the -D option) and the following echo $! >$DIR/httpd.pid writes mini_httpd's process id into the pidfile.

Add your own code to the script to support services you need depending on their type.

Installing otg-disk

You can run otg-disk simply on demand from the command line (or from within a shell script):

./otg-disk

Added the parameter _force_ if you want to bypass the text for USB ethernet.

If you want otg-disk as a permanent service you can install it with

sudo ./otg-disk _install_

otg-disk will create a systemd unit file in /etc/systemd/system and register it. That's all. As you may have guessed it, the unit file is created on the fly as a here document inserting the required parameters.

After that

Of course you can start, stop or restart the service with systemctl too.

If you want to run the service without the usb0 condition on your W-LAN connection you would use

sudo ./otg-disk _install-wlan_

which skips the test.

Uninstalling is as easy as installing:

sudo ./otg-disk _uninstall_

does that.

Implementation notes

otg-disk implements only a few system functions out of the box: Rebooting and shutting down are important to handle the computer properly even if ssh is not available. Other services (e.g. more network servers) must be added manually to the script. That's ok because it keeps otg-disk self contained but make it difficult to port functions from one computer or user to another.

Then otg-disk's process handling with pid files is somewhat old fashioned. systemd is something that comes immediately into mind but has the disadvantage that you have to modify your system configuration in some way for that (beyond installing otg-disk as service). And I definitely wanted to avoid that. (You can get rid of the installed programs with sudo apt purge nmbd smbd inotify-tools.) uinit is another option that would fit better to otg-disk than systemd but I wanted to keep otg-disk as simple as possible for the user: Not touching the system configuration for systemd or downloading something and running dpkg for uinit.