Using ffserver to transcode live radio and broadcast to unsupported clients (Chromecast, .m3u8)

Google Chromecast can be easily configured to play audio or video files over HTTP. Using PyChromecast, one can play a publicly-accessible audio file by providing its URL.

media_controller.play_media("http://example.com/audio.mp3", 'audio/mp3')

However, things start to go wrong when trying to play HLS streams, which have the .m3u8 file extension.

Enter ffserver

ffserver is (or rather was) an easy way to take a input feed and transcode it to one or may output streams while being provided with the ffmpeg binaries. Since ffmpeg can already decode .m3u8 HLS streams, it can be used directly as an input feed to ffserver. Given the right configuration, on can then expose this feed as a streaming mp3/ogg/aac file which can be understood by the Chromecast, or any other dumb player which can play a file from an HTTP stream.

ffserver serves ordinary audio streams (.ogg, .mp3, …) over HTTP.

N.B. ffserver has been officially deprecated in 2016 and subsequently dropped from ffmpeg binaries in 2018. Ubuntu 18.04 repositories serve ffmpeg 3.4.6 which still includes ffserver.

Configuration

ffserver’s configuration itself is pretty straightforward. We both define an input feed and an output stream. Multiple streams can be defined for the same feed.

/etc/ffserver.conf
See https://github.com/keotl/ffserver-live-radio-mirror-example for pastable code and a Dockerfile!

Running ffserver should now start an HTTP server, which serves our stream1.ogg on http://localhost:5000/stream1.ogg .

See this GitHub Repository for a complete working example.

Understanding the HTML Audio Element Buffered Regions

The HTML audio element exposes multiple properties and methods which prove to be useful for making a reactive web music player. In addition to the usual play/pause controls, the <audio> tag has a currentTime, a duration, and a buffered attributes which can all be used to get the playing media state in real time using a bit of JavaScript code.

currentTime and duration

The currentTime and duration attributes are pretty straightforward. currentTime returns the playback position from the audio file start, in seconds, as a float. Similarly, duration is a float corresponding to the total media length. One can then simply convert to a more natural time format using a bit of formatting code.

export function formatDuration(floatSeconds) {
  const durationMillis = floatSeconds * 1000;
  const timeInSeconds = Math.floor(durationMillis / 1000);
  const minutes = Math.floor(timeInSeconds / 60);
  const seconds = timeInSeconds - (minutes * 60);
  const formattedSeconds = seconds > 9 ? `${seconds}` : `0${seconds}`;
  return `${minutes}:${formattedSeconds}`;
}

buffered regions

The HTML audio element internally uses multiple buffered regions, which live independently. Each buffered region start and stop times can be accessed using the following JavaScript lines.

audioelement.buffered.start(<buffered_region_id>)
audioelement.buffered.end(<buffered_region_id>)

Initially, the browser creates a first buffered region (numbered 0) which is enough to play the first few seconds. As the playback approaches the end of that region, a new part of the media file is automatically requested.

If the user happens to seek the media to an unbuffered region, the browser creates a new buffered region starting at that point. This means that, to properly reflect the next buffered zone, our application has to now show the buffered region #1 in gray. Then, when the user rewinds back to a time inside the first buffered region, our 0th region gets reused. Note that the browser likes to rearrange these ranges in the background, so hardcoded values will definitely become an issue.

Here is a simple JavaScript/VueJS snippet which finds the present buffered region and updates the “buffered but unplayed” region in our application.

updateSeekBar() {
      const currentTime = this.$refs.audiotag.currentTime;
      const totalDuration = this.$refs.audiotag.duration;

      const timeRangeLength = this.$refs.audiotag.buffered.length;
      let currentBufferRange = 0;
      for (let range = 0; range < timeRangeLength; range++) { 
             const rangeStartTime = this.$refs.audiotag.buffered.start(range); 
             const rangeEndTime = this.$refs.audiotag.buffered.end(range); 
             if (currentTime >= rangeStartTime && currentTime < rangeEndTime) {
                 currentBufferRange = range;
         }
      }
      this.bufferedPercentage = (this.$refs.audiotag.buffered.end(currentBufferRange) - currentTime) / totalDuration * 100;
    },

Handling Partial Content Requests (HTTP 206) for audio streaming

Web content streaming requires sending partial files between a server and a client, most often a web browser. This is a strict requirement for controlling audio playback using an HTML audio tag. While a simple HTTP 200 response containing the whole file will work for most use cases, it will not allow proper playback control, as is to be expected of a respectable music streaming app. 

Request

First, the browser requests a partial file by including the Range header, which encodes the position in bytes. The first word refers to the allowed range type, bytes in our case.

bytes=<start_offset>-<end_offset>

Response

Next, the server should reply that :

  1. it accepts file ranges, using the Accept-Ranges: bytes header;
  2. it is sending an incomplete file, using the 206 HTTP status code, containing a specific number of bytes with the Content-Length header;
  3. and that it is sending the requested byte range, in the Content-Range header.

The Content-Range header should contain the first byte index, the last, and the total file size.

Content-Range: bytes <start>-<end>/<total>

Note that the last fragment’s “end” offset should be exactly one less than the total file size.

Example

Below is a quick and dirty example for sending a partial file in python.

if request.headers['HTTP_RANGE']:
  total_size = os.path.getsize(filename)
  http_range = request.headers['HTTP_RANGE']
  start = int(http_range.split("=")[1].split("-")[0])
  end = start + 2000000 if http_range.split("-")[1] == '' else int(http_range.split("-")[1])
  
  if end > total_size:
      end = total_size - 1
  
  with open(filename, 'rb') as f:
    f.seek(start)
    body = f.read(end - start)
  return Response(206, {"Content-Type": "application/octet-stream",
                        "Accept-Ranges": "bytes",
                        "Content-Range": "bytes {}-{}/{}".format(start, end, total_size)},
                  body)

 

Making An SMS Chatbot #2 – Simple Natural Language Processing

Now that we have a way of sending and receiving SMS messages, let’s put it to use with some natural language processing!

Tokenize and Stem

The first step towards any sort of natural language processing is normalizing the input, as to make it easier to understand, or at the very least, manipulate. In our case, we want to split the input into words, and then remove any alteration, as to bring them back to their most simple form, or stem. That means removing plural signs and verb conjugations, amongst other things. For this, I used the Apache OpenNLP library.

Algorithm

Now that our input is “tokenized”, it can be used to issue commands. To keep things simple, the chatbot will respond to commands with a predefined syntax. Possible commands will be registered with an expected word sequence, and will then be invoked when the corresponding sequence is received. When the program starts, a command tree is created, which can then be navigated, word by word, to find the associated method call.

The input message “Say hello.” will cause a line to be printed to the console.

To allow for more variation, all extraneous words are ignored. For instance, “Keep saying a very grateful hello forever.” will resolve to the same method.

Parameters

The algorithm itself is quite simple, but can be expanded to make it a bit more powerful. For instance, passing parameters could be a nice addition. If those could be properly typed, that would be even better.

Using reserved keywords when registering a command, I can define parameters to be passed onto the invoked method. These keywords are user-defined, therefore completely customizable. At the moment, I have included most basic types, namely Integer and String, but in the future, one might see a command similar to “track package <UPS-tracking>”, or even “what is $ <int> in <currency>?”.

Here is what a command declaration looks like in Java.

Handling User Identities

Since all commands are tied to a phone number, we can use it to do some basic user authentication. In fact, it would be even nicer to have users belong into “groups”, or “roles” which then determine the commands they are allowed to access. When a user invokes a forbidden command, it can simply be ignored.

Users are saved to a MySQL database, using Hibernate ORM.
To make manipulating permissions easier, I added a CommandContext object which can be used in any command method to get contextual information, which includes the user information.

“Conversational” Commands

Repeating several times in a row “Say hello 5 times” can become somewhat tedious. A “say that again” command could be useful.

In fact, it would be great if our chatbot, like most modern assistants, had some contextual or conversational awareness. That way, we could define commands to be only accessible after a particular relevant command has been invoked. For instance, “Yes” and “No” commands could be enabled after a question, as part of the ongoing conversation.

This is achieved by using a “@Conversational” annotation on the new command, which defines the method call after which it should be enabled. Then, when a new message is received, the conversational commands are checked first for a match. This can be used as an effective way of “masking” default commands depending on the context.

Further Improvement

As-is, the bot “works”, but does not have any useful functionality. You can say “hello” to introduce yourself, which will create a new user in the database. It can remember names too, which could theoretically be used to customize messages. There are, however, several issues which should be addressed before going further.

  • The database access is NOT thread-safe. The bot works, as long as there is only one concurrent user.
  • The actual command tree-searching methods are getting quite complicated. The class itself could definitely be more finely divided to improve readability.
  • The goal is to, someday, listen on multiple communication endpoints. For instance, the bot could be used as a Skype or Messenger bot. This should definitely be feasible in its current implementation. We could probably go a step further, and use a proper SMS service, instead of one I hacked together using a spare iPhone 3GS.
  • Right now, defining synonyms, or aliases for commands is quite clunky. It requires creating a brand new method, with its own @CommandHandler annotation. Allowing arrays or some other way to define aliases would be very useful. The same can be said of conversational commands.
  • The bot needs a friendly name!

See on GitHub.

Minecraft Discs : Dynamically-loaded Custom Music Discs From YouTube Videos

As usual, the code is on GitHub.

A while ago, I had the idea of a simple Minecraft mod which would allow the creation of custom music discs in Minecraft using YouTube as the primary way of distributing and synchronizing the actual music tracks amongst players. All of this, while allowing any player on the server to add their own favorite music tracks.

The Forge

The actual Minecraft mod is quite simple. It reads a simple JSON-formatted list of tracks, and register them as items with Forge Mod Loader, giving them a game ID. At the same time, they get an item texture assigned, which is simply one of the existing record item textures.

Building Resources

To get the music tracks to play, the mod uses a separate resource pack, built by grabbing YouTube videos, and extracting the audio using ffmpeg. I made a simple java program to parse the JSON track list and download the tracks, while creating the required directory structure for a Minecraft resource pack. At this step, track titles also get added into the game.

Keeping Clients Coherent

Since the primary use case of this mod is multiplayer gameplay, we needed a way of keeping clients coherent, while still allowing a frequent additions. I came up with a simple CRUD-style application hosted on a Heroku container, which would serve a properly formatted track list, and offer a simple form for user to add new songs, all of that using a free-tier database. The only downside to this approach is that the container shuts itself off after 30 minutes of inactivity, then requiring roughly 25 seconds to serve the next request. This causes the game client to freeze in the loading screen, while waiting for a reply. This could be easily fixed by stepping to a more premium usage tier, but the very light usage mostly makes this a non-issue.

Making an SMS Chatbot #1 – Exposing a REST API on an iPhone 3GS for sending and receiving SMS

This is a fun little project I started while toying around with a spare iPhone. All mentioned code can be found on my GitHub.

Setup

For this project, I will be using a jailbroken iPhone 3GS running iOS 6.1.6, the latest it can handle. The nice thing about these older devices is that they can be easily restored to any compatible iOS version, without any trouble. Jailbreaking then allows us to get really creative with the software running on the machine. I used the redsn0w jailbreak for the new-bootrom iPhone 3GS, with the p0sixspwn package to make it an untethered jailbreak.

Going Online

The most important part of a web API is, quite obviously, the web part. For this purpose, I used the ios-webstack.tk Cydia repository which provides a “one-tap solution” for installing a complete WebStack on a jailbroken iOS device. The repository actually contains multiple network service packages, such as an SMTP server, and a fully-fledged WordPress installation. The part we’re more interested in, however, is the ios-lighttpd-php-mysql package, which installs, amongst other things, an HTTP daemon and a convenient preferences panel in the settings app to enable or disable the service.

I configured httpd to execute CGI scripts, and set a filetype association for .sh and .py scripts to their respective interpreters. That way, when the server gets a request for a python file, it will instead execute it and return the console output as a raw HTTP response. This will allow us to make quick and dirty scripts for running on the very resource-limited hardware that we have.

Sending Messages

For sending out SMS messages, I could not find any built-in or third-party options that were dedicated to operating the messaging app from the command line. That said, I did find a quite obscure and undocumented use for the biteSMS app that will allows me to send messages from a shell script. The app itself is riddled with ads, but luckily we won’t be using its GUI. Once installed, we can call it from a shell prompt to send a text message!

/Applications/biteSMS.app/biteSMS -send -carrier 1234567890 <my message here>

 Receiving Incoming Transmissions

Now comes the fun part. Ideally, we would like a way of triggering an action whenever an SMS is received. In the best-case scenario, we would be calling a shell script directly. Worst-case, we would end up catching some UI notification, which would be somewhat impractical, be we could work with that.

I looked into Activator triggers for incoming messages, sadly the triggering action was not as reliable as I would have liked. The iPhone ended up missing many text messages, or only triggering when multiple unread messages had started piling up. In the process however, I did find ways of triggering UI system functions, like capturing a screen shot, from the command line, which is nice, but not technically useful for the task at hand.

 How the iPhone stores messages

While digging around, I stumbled upon the internal SMS database, which is simply an sqlite3 database. The .db file itself is located in /var/mobile/Library/SMS/sms.db. In there, we find a “chat” table containing the conversation metadata, and a “message” table, which contains the actual text messages. It is then a matter of writing an SQL query to grab that data.

SELECT m.rowid, c.chat_identifier,m.is_from_me,m.date,m.text FROM message m INNER JOIN chat_message_join cmj ON cmj.message_id=m.rowid INNER JOIN chat c ON cmj.chat_id=c.rowid WHERE cmj.message_id=:id

To trigger an action when a message is received, I wrote a little background script which polls the database every second, checking whether a new entry has been created by looking at the highest id assigned. When a change is detected, the script makes an HTTP request to my computer, which runs the backend software.

The iPhone, like OS X, uses the plist format with launchd to control system services. The syntax is somewhat more esoteric than the more common systemd, but it gets the job done anyway. The service files are located under /Library/LaunchDaemons, and can be enabled using launchctl.

Other comments

  • To prevent the iPhone from turning off its Wi-Fi connection after going to sleep, I used the Insomnia Cydia tweak.
  • Instead of going through the process of installing a recent python version, which would have included proper JSON serialization support, I formatted the HTTP response manually by concatenating various parts of the SMS data. Using proper JSON formatting would definitely be better, especially in the long run, but I did not really want to go through the trouble of installing multiple python versions on the iPhone.

I now have a way of sending and receiving SMS by sending a few JSON-formatted requests. That should definitely come in handy later…

Retro Mac Adventures #2 – Linux on the ’06-’07 MacBook

Last time, I was able to get up and running hardware-wise. Next, I will be installing a Linux distribution to get modern software on the device. I will be using the 64-bit version of Ubuntu 16.04 LTS which should run just fine on the new Core 2 Duo board.

Boot Hurdles

Early Intel Macs usually do not play nicely with third party operating systems. Even though the CPU itself is 64-bit capable, the EFI boot subsystem requires 32-bit binaries. As-is, bringing up the boot-selection menu using the option key will show the Ubuntu installation disc labelled “Windows”. Selecting it, however, will hang the system. Using an external USB hard drive or thumb drive yields similarly unsatisfactory results. Those computers will only boot OS X from a USB device.

Installing rEFInd

Using an external OS X installation, setting up the rEFInd Boot Manager first will make it easier later on when we are getting to a bootable state. rEFInd provides a nice dynamic boot menu, which detects bootable media from all connected devices, regardless of the OS. Sadly, at least on early iterations of the MacBook line, the Ubuntu installer cannot be started directly this way, which I found out in the process.

Getting “Creative” with the Installation Process

Even though the Mac stubbornly refuses to boot the installation media, I will not be defeated! Let’s list our options :

  • Use 32-bit Ubuntu;
  • Look for another Linux distro, hopefully one which boots correctly on the Mac;
  • Give up and write a blog post about it.

The first option seems like a nice tradeoff… at first glance, that is. I know for a fact that it simply works, having tried it before. However, many modern softwares are dropping 32-bit support (*cough* chrome *cough*), and many things simply feel slower. At this point, we have set out on a quest to install 64-bit Ubuntu, and settling for anything less would just feel like laziness, thereby leaving me with a taste of defeat and overall disappointment.

On the Surface

Everything wrong with DongleScape in a single picture.

There is another way however. Come closer, as I tell ye the tale of external installation. What I ended up doing the first time I tried installing Linux on a MacBook, was to connect its hard drive to a working host system, and performing a “chrooted” installation using an arch linux bootable media. However, unlike Arch Linux, Ubuntu has a nice bootable graphical installer which sounds easier than a console installation. (At that time anyway…) So, using the graphical installer, on a Surface Pro 1 I had lying around, I was able to install the OS directly on the SSD, while incurring a still manageable mess of wires.

DongleScape \ ˈdäŋ-gəl-ˌskāp \, noun. The adventure of using an immature connector standard. Its adepts can be recognized by the unearthly amount of adapters in their bags. Most fervent followers will carry more than required for their needs, as an exercise of self-discipline, part of their ascetic lifestyle.

 

Getting it to Work

rEFInd does not detect 64-bit EFI binaries

Installing from another computer usually causes its own lot of issues, which can range from mildly rage-inducing, to simply annoying. The Surface Pro 4, using a newer 64-bit EFI boot system, caused the wrong type of GRUB binaries to get installed on the drive. Using an existing installation on a similar MacBook, I was able to mount the freshly-installed Ubuntu and copy some 32-bit EFI binaries. While they were unsuitable for the actual drive, they still allowed me to boot to a GRUB rescue shell, from booting was possible. From there, I reinstalled GRUB, once again, to generate proper EFI stubs for the system.

The End Result

This was probably an overcomplicated mess, but whatever, it works. The result is a nice, slick, black MacBook which is still usable in 2017.

Retro Mac Adventures #1 – A Fine Addition

The Find

Recently, I came across an eBay auction for a 2006 black MacBook which was being sold for parts. From the pictures, the case seemed in quite good shape. A bit grimy, sure, but nothing a bit of rubbing alcohol could not solve. For a mere 11 dollars, a surprising amount of computing can be had on these days. There it is, ticking all the usual marketing boxes :

  • $11.50 ✔️
  • Glowing Apple logo  ✔️
  • No battery ✔️
  • No cable ✔️
  • No power ✔️
  • No working ✔️
  • No return ✔️

What a steal!

A few weeks later, it arrived in the mail, and I was in for some cleaning duty, especially around the keyboard area.

Individually removing and cleaning the keycaps seems to have done a decent job at removing what looks like food leftovers and other unidentified organic matter. I was not able to completely remove that musty smell though.

Turing It On

Unsurprisingly, powering the machine on using my trusty MagSafe 1 did not cause the satisfying boot chime to play. Toying around with various boot key combinations led to the conclusion that the unit had no RAM whatsoever, solving that part of the mystery. New RAM, new hard drive, and we’re rocking like it’s ’06.

At this point, we can confirm that most of the unit is still functioning correctly.

  • Display ✔️
  • Disk drive ✔️
  • Motherboard ✔️

However, this does not account for the random shutdowns and unusually loud fan, which likely indicates a thermal issue of some sort. This calls for some further investigation.

Paranormal (Thermal) Activity

Once loaded into OSX, investigating requires reading thermal sensor data, which proved surprisingly difficult without an internet connection and on a 10-year-old deprecated OS. Luckily, I found one such tool, which would allow me to look at that data for free and without some sort of trial period.

Well, looks like we found a culprit! That “Main Heatsink 1” sure looks out of place with its toasty 54 degrees. In fact locating the actual problematic probe was a simple task, requiring a basic poking tool to poke around while doing an open-brain surgery on the patient.

127 happens to be the largest integer that can be stored on 7 bits.

Oddly, unplugging and reconnecting the probe connector on the logic board was enough to fix its readings. Here it is, running a whisper-quiet, poor-man’s CPU stress test.

Sometimes, it almost looks as though I know what I am doing!

Performance Upgrades

Since part of the plan is to use this Mac as a daily driver, 2 GB of RAM and a 10-year-old 80 GB hard drive are not going to cut it. For the boot drive, I have gone with the cheapest SSD I could find. However, that drive did not come with a spacing adapter, leaving it to rattle around inside the case. I had to, somehow, “specially improvise”.

The unit came with a Core Duo T2500, which is a 32-bit only processor. This can be quite inconvenient, and can severely limit the performance in some cases. Since I also happen to have quite a few Core 2 Duo boards lying around, doing a motherboard swap was a somewhat quick and painless way of getting that extra kick when needed. It also allowed 3

GB of memory, up from 2 GB.

 

 

 

 

 

To remedy the “no battery” situation, I was able to track down another eBay listing, this time for a brand new, sealed, genuine battery for the laptop for half ifixit‘s price. Needless to say, I was impressed. That 1 cycle count was quite a satisfying sight on a 9-year-old battery.

 

That leaves me then with a fully-functioning black MacBook, which goes nicely with the white one I have been using daily for the last year.

Next, I will be installing Linux on the thing, to make it truly usable in 2017.