biking2, Architecture and API

The main purpose of biking2 is keeping track of bicycles and their milages as well as converting Garmin Training Center XML (tcx) files to standard GPS Exchange Format (GPX) files and storing them in an accessible way.

In addition biking2 is used to study and evaluate technology, patterns and frameworks. The functional requirements are simple enough to leave enough room for concentration on quality goals.

The application must only handle exactly one user with write permissions.

Most bike aficionados have problems understanding the question "why more than one bike?", the system should be able to keep track of everything between 2 and 10 bikes for one user, storing 1 total milage per bike and month. All milages per month, year and other metrics should be derived from this running total, so that the user only need to look at his odometer and enter the value.

The application should store an "unlimited" number of tracks.

The images should be collected from Daily Fratze, the source are all images that are tagged with "Radtour". In addition the user should be able to provide an "unlimited" number of "gallery pictures" together with a date and a short description.

The following lists contains the most important personas for this application

A passionate biker uses biking2 to manage his bikes, milages, tracks and also visual memories (aka images) taken on tours etc. He also wants to embed his tracks as interactive maps on other websites.

Daily Fratze provides a list of images tagged with certain topics. biking2 should collect all images for a given user tagged with "Theme/Radtour".

GPSBabel is a command line utility for manipulating GPS related files in various ways. biking2 uses it to convert TCX into GPX files. The heaving lifting is done by GPSBabel and the resulting file will be managed by biking2.

The user may want to embed (or brag with) tracks on arbitrary websites. He only wants to paste a link to a track on a website that supports embedded content to embed a map with the given track.

biking2 is broken into 2 main components:

The api runs on a supported application server, using either an embedded container or an external container. It communicates via operating system processes with GPSBabel on the same server.

The connection to Daily Fratze is http based RSS-feed. The feed is paginated and provides all images with a given tag but older images may not be available any more when the owner decided to add a digital expiry.

Furthermore biking2 provides an oEmbed interface for all tracks stored in the system. Arbitrary websites supporting that protocol can request embeddable content over http knowing only a link to the track without working on the track or map apis themselves.

The frontend is implemented with two different components, the biking2::spa (Single Page Application) is part of this package. The spa runs in any modern web browser and communicates via http with the api.

The following diagram shows the main building blocks of the system and their interdependencies:

I used functional decomposition to separate responsibilities. The single parts of the api are all encapsulated in their own components, represented as Java packages.

All components depend on a standard JPA EntityManager and some on local file storage. I won’t go into detail for those blackboxes.

Additional configuration apart from properties lives in the config package:

The support package contains technical infrastructure classes while the shared package contains application wide messages. Those packages are used as supporting packages:

The database package contains generated code for accessing our database via jOOQ:

biking2 is a datacentric application, therefore everything is based around an Entity Relationship Diagram (ER-Diagram):

Those tables are mapped to the following domain model:

biking2 uses an H2 database for storing relational data and the file system for binary image files and large ascii files (especially all GPS files).

During development and production the H2 database is retained and not in-memory based. The location of this file is configured through the biking2.database-file property and the default value during development is ./var/dev/db/biking-dev relative to the working directory of the VM.

All access to the database goes through JPA using Hibernate as provider. See the Domain Models for all entities used in the application.

The JPA Entity Manager isn’t accessed directly but only through the facilities offered by Spring Data JPA, that is through repositories only.

All data stored as files is stored relative to biking2.datastore-base-directory which defaults to ./var/dev. Inside are 3 directories:

The default user interface for biking2 which is packaged within the final artifact is a Single Page Application written in JavaScript using Angular JS together with a very default Bootstrap template.

For using the realtime location update interface, choose one of the many MQTT clients out there.

There is a second user interface written in Java called bikingFX.

JavaScript and CSS dependencies are managed through Maven dependencies in form of webjars wherever possible without the need for brew, npm, bower and the like.

Furthermore biking2 uses wro4j together with a small Spring Boot Starter to optimize JavaScript and CSS web resources.

wro4j provides a model like this:

This model file is filtered by the Maven build, version placeholders will be replaced and all resources, in webjars as well as inside the filesystem, will be available as biking.css and biking.js.

How those files are optimized, minimized or otherwise processed is up to wro4js configuration, but minification can be turned off during development.

biking2 relies on Spring Boot to create all necessary beans for handling local transactions within the JPA EntityManager. biking2 does not support distributed transactions.

biking2 only provides a stateless public API, there is no session handling.

biking2 offers security for its API endpoints only via HTTP basic access authentication and in case of the MQTT module with MQTTs default security model. Security can be increased by running the application behind a SSL proxy or configuring SSL support in the embedded Tomcat container.

For the kind of data managed here it’s an agreed tradeoff to keep the application simple. See also Safety.

No part of the system has life endangering aspect.

biking2 uses an internal Apache ActiveMQ broker on the same VM as the application for providing STOMP channels and a MQTT transport. This broker is volatile, messages are not persisted during application restarts.

Datatypes and ranges are checked via JSR-303 annotations on classes representing the Domain Models. Those classes are directly bound to external REST interfaces.

There are three important business checks:

Errors handling to inconsistent data (in regard to the data models constraint) as well as failures to validation are mapped to HTTP errors. Those errors are handled by the frontends controller code. Technical errors (hardware, database etc.) are not handled and may lead to application failure or lost data.

Spring Boot configures logging per default to standard out. The default configuration isn’t change in that regard, so all framework logging (especially Spring and Hiberate) go to standard out in standard format and can be grabbed or ignored via OS specific means.

All business components use the Simple Logging Facade for Java (SLF4J). The actual configuration of logging is configured through the means of Spring Boot. No special implementation is included manually, instead biking2 depends transitively on spring-boot-starter-logging.

The names of the logger corresponds with the package names of the classes which instantiate loggers, so the modules are immediately recognizable in the logs.

Spring Boot offers a plethora of configuration options, those are just the main options to configure Spring Boot and available starters: Common application properties.

The default configuration is available in src/main/resources/application.properties. During development those properties are merged with src/main/resources/application-dev.properties. Additional properties can be added through system environment or through an application-*.properties in the current JVM directory.

During tests an additional application-test.properties can be used to add or overwrite additional properties or values.

Those are the biking2 specific properties:

Only supported language is English. There is no hook for doing internationalization in the frontend and there are no plans for creating one.

biking2 replaced a Ruby application based on the Sinatra framework. Data was stored in a SQLite database which has been migrated by hand to the H2 database.

The project contains JUnit tests in the standard location of a Maven project. At the time of writing those tests covers >95% of the code written. Tests must be executed during build and should not be skipped.

The application can be build with Maven without external dependencies outside Maven. gpsbabel must be on the path to run all tests, though.

Popular JavaScript mapping frameworks provide easy ways to include geometries from GPX data on maps. Most Garmin devices however record track data in TCX format, so I needed a way to convert TCX to GPX. Both formats are relatively simple and in case of GPX good documented formats.

biking2 uses GPSBabel for the heavy lifting of GPS related data. The project contains a README stating that GPSBabel must be installed. GPSBabel can be installed on Windows with an installer and on most Linux systems through the official packet manager. Under OS X it is available via MacPorts or Homebrew.

biking2 needs to store "large" objects: Image data (biking and gallery pictures) as well as track data.

I opted for local file system because I didn’t want to put much effort into evaluating cloud services. If biking2 should runnable in cloud based setup, one has to create an abstraction over the local filesystem currently used.

biking2 was not received as database centric in the beginning. Hibernate entities had been modelled, the database was setup with Hibernates automatic DDL. Analytic functions had not been used, but computation has been done in memory.

Use a database centric approach as described in "Live with your SQL-fetish and choose the right tool for the job"

In late 2019, that decision has been implemented. Flyway has been introduced to create tables and other migrations via SQL scripts. Based on a database that is actually under our control, we generate a jOOQ schema, on which all SQL generation for computing statistics is done. For more information, have a look at the whitebox view of the statistics modul.

By using JaCoCo during development and the build process ensure a code coverage of at least 95%.

The architecture should be designed in such a way that algorithms depending on external services can be tested without having the external service available. That is: All external dependencies should be mockable.

Example: FetchBikingPicturesJob needs a resource containing a RSS feed. Retrieving the resource and parsing it are at least two different tasks. Fetching the resource through a separate class DailyFratzeProvider makes testing the actual parsing independent from a HTTP connection and thus relatively simple.

To keep track of all milage including time not spent on persistent bikes, the user can store assorted trips. A trip is a distance covered at a certain date.

The application provides a nice banner during startup in the logs. It would be a shame not providing an API for that. To retrieve your ASCII art banner, just juse the following API: