Reckoner Update September 2023

Progress was actually mad this month! I am inching ever closer to releasing the application.

šŸ”„ Sync Updates

So I will not be doing peer-to-peer synchronization for the first iteration of Reckoner. I have instead gone back and re-implemented PocketBase synchronization.

šŸ’± Peer-to-Peer Using WebRTC

So, I was bummed with the response of the peerdart library to the issues that cropped up with the application. Even as I write this, there have been no replies to the Github issues raised by other people who noticed the same issue.

Well, not to be deterred by this, I decided to implement the WebRTC client for Peer.js in Dart myself! One of the things I immediately noticed on pulling the Peer.js client code is that peerdart essentially took the Peer.js client repo and made minimal changes to convert it into Dart code. While I expected a bit of copy and paste, this was essentially copy all and modify until working.

Building it out greenfield, I was able to get a working communication between two peers! Not only that, it is far leaner than what is implemented in peerdart. It was at then that I encountered another issue in the core flutter_webrtc library. As of Flutter 3.13, it appears they are cracking down more on when and where people run code. There are errors about ā€˜channel sent a message from native to Flutter on a non-platform threadā€™ (see #1413 and #1428). As I can best tell, they want all platform threads to be initiated from the main (UI) dart isolate (thread) or from an explicit background isolate and donā€™t want you to use native threading techniques. Iā€™ve seen this pop up when initializing a WebRTC data connection or when sending a message on the data channel.

I am not going to try and solve the flutter_webrtc issue. While I certainly am proficient enough to figure out Flutterā€™s WebRTC libraries and utilize them, I am not proficient enough to perform the work necessary to fix this. As of right now, I will shelve the peer synchronization work and focus on server based approaches for data synchronization.

šŸ–„ļø PocketBase Server

I was able to quickly resurrect the PocketBase synchronization code and get it successfully working! This was done in the midst of the WebRTC investigation. However, I didnā€™t like the way I was saving and arranging data on the PocketBase implementation.

One of the benefits of the PocketBase implementation is that I can use it as a template for a Reckoner software-as-a-service (SaaS) synchronization model. The way I previously designed it was three tables: transactions, budgets, and time-invariant data with each row being an individual item for the transactions and budgets table and the time-invariant table having one giant item with all of the other data which isnā€™t attached to a date (accounts, categories, reports, etc). This was simple to manage, but could pose problems with the size of the time-invariant row. This row is essentially the entire database minus transactions and budgets.

With that in mind, I went back to the drawing board. What Iā€™ve finally come up with is a single table which has two data columns: json (or encrypted string of data) and a flag determining if time-invariant or not. The client can just json deserialize the data into the appropriate class (and unencrypts if encrypted). The different data bundle types are for an individual organization (grouping of accounts), category group (grouping of categories), reports & settings, transactions, and budgets. The latter two are time varying. This is slightly more complex, but does make it so that I donā€™t have just one giant piece of data for the time invariant items.

This does have one problem that it is easier to possibly correlate data to a user since you can get more information about the data theyā€™re storing. Say you know they like to use a lot of different category groups and report groups. You could query users on the server with a high number of time-invariant rows and window down the list of candidates without decrypting any data. Although, you could do something similar in the last iteration of this by just looking at the size of the data stored in the single time-invariant row for the user.

There is no perfect solution. For the highest privacy, a person would probably use a single device and create manual database backups in an offline encrypted usb drive. The cloud synchronization SaaS service will not appeal to these users, but a self-hosted PocketBase instance may depending on threat model. The cloud service will be there for your non-tech savvy grandpa who just wants something to work or for the low threat model person who doesnā€™t want to self-host.

ā³ So How Long Till Release?

I want to have Reckoner v1.0.0 out by the end of the year. One of the items I havenā€™t outlined here is my plan for the versioning scheme for Reckoner. I plan for the first number to be the database version. So anytime I need to add/modify/remove a table, the version number will be bumped.

Whatā€™s Next

I am going to remove support (at least from me) for FDroid. I will have a future article on why and what I will recommend to people for getting my app. If anything, I may ask it to be added on the IzzyOnDroid F-Droid Repository instead for people on FDroid.

General Tasks

Version 0.2: Multi-Device Synchronization

Version 1.0 (Future)


← Reckoner Update August 2023

Reckoner Update October 2023 →