Reckoner Update June 2023

The progress was pretty incremental this month as well. I’ve made some pretty big infrastructure backend changes.

☁️ Sync Updates

I had some difficulties at first setting up PeerDart. Originally I was trying to reuse some of the existing synchronization code I wrote for PocketBase, but I ultimately needed to write wholly new code to handle the case of peer-to-peer communication. I now have the ability for two peers to initiate communication with each other, and exchange device information.

The synchronization infrastructure will be end-to-end encrypted. I am currently using a symmetric key exchange instead of a public-private key exchange since it is simpler to use symmetric keys. I might consider making it more complicated in the future and looking at per-device keys instead of one key for each device. This would increase security.

I also need to think about how to handle adding the third or more devices and letting other devices know about that device. This will be an area of focus for next month.

🧑‍💻 Refactoring

Or how I learned to stop worrying and love MVC.

Background

To provide some background on the latest work, I need to let you know about one of the members of my Brain Trust. He’s the OG member when I wanted to have an excuse to talk more often, but needed a structured way to do so. Let’s call him Frood. He has a background in human factors for man-machine interfaces (think layout of screens and controls for a process) and some background in managing development projects. One of the first things he hammered on when I was outlining the infrastructure is what is the Model, View, and Controller for Reckoner.

Original Organization

I resisted the MVC pattern at first and looked up other organizational structures which people were using for organizing Flutter projects. I can’t find the reference now, but I settled on a folder structure based on what the code did. Before the refactor I ended up with a folder structure like this which roughly corresponded to the following in each folder.

The issue was that concerns were poorly separated. Widgets were often directly accessing the ReckonerDb object to get a data stream from a DAO query. The worst case was the category_group.dart which had both presentation of the data, mutation of data to alternative classes and models, and logic around setting balances and doing computations. Another pain point was that some views had to deal with 3 different classes for the same data type (Transactions - SQL ORM model, TransactionInfo - immutable data with relationships to other class entities, and TransactionSave to handle editing and saving of the transactions) while other just simply used one class like the currency edit.

One common thread is that I tended to code my view and model with a backend mindset. The *Info classes would often have helper functional useful for presenting the data. Similarly, the *Save classes all had a .save() function which would do validation of the *Save model state and save it to the database. However, a widget could be working with a *Info object, *Save object, the ORM classes, or occasionally all three! Grouping data and the logic that acts on the data is generally a good practice in the backend, but has made the code unwieldy in the frontend.

New Organization

I first tackled this by seeing how I could eliminate the *Info and *Save classes. After several iterations and some tests later, I ended up with the following folder structure.

This may seem more complicated just looking at the folder structure, but has greatly simplified separation of concerns. I strive so that files from different root folders shouldn’t need to import anything more than one directory deep. Thus the view should only need to import the necessary classes from the model directory for the models it is touching and the controller class for any data loaders and logic. Also, the controller shouldn’t ever load anything from the view folder and the same for the model with the controller and view. Additionally, I have been actively moving logic to the controller folder so the view folder is strictly presentation.

I also stopped creating unit tests about a year ago originally since I was spending too much time maintaining them due to changes. I’ve come to realize now that it is much easier to test and maintain the unit tests when I’ve stopped intermixing the logic and presentation. I’ll start to consider writing tests again and looking at my testing coverage for the controller and view folders. The goal is to have nothing, or next to nothing to test in the model as all of the logic is moved to the controller folder.

🧑‍🏫 Lessons Learned

Conventions are conventions for a reason. However, I tend to learn a few things the hard way 🤦. Frood said it best when I first announced the conventions change.

While this may seem painful, the value of architecture only begins to shine when you reach a certain level of complexity. Congrats, Reckoner has breached the threshold!

This is very true. When I was first programming the simple bits, currency and accounts at the time, it seemed like separating these concerns out just made it more work and more complicated. Now that the application is sufficiently complicated, I see how conventions like MVC really help.

Don’t confuse this for me saying MVC is the be-all, end-all of application architecture either. There are lots of good alternatives to MVC for frontend application development and MVC makes no sense for backend development. In fact, a lot of my background was in backend development where the common patterns I encountered were object oriented coupling of data with logic specific to that data and functional where functions would transform data as needed. Even the time I spent doing frontend development was atypical as we were using Visual Basic for the frontend and the code was action-procedural.

It also helped for me to have the concept re-contextualized and mentally relabeled. I think of the view as functional and a function of the input state, the model as pure data (as much as possible), and the controller as all of the logic necessary to glue the other two. I’m not sure why this specific reframing helps me, but it’s what finally got me to understanding the concept and its importance.

What’s Next

More tasks have been added to version 0.2. Still, good progress for this month.

General Tasks

Version 0.2: Multi-Device Synchronization

Version 1.0 (Future)


← Reckoner Update May 2023

Reckoner Update July 2023 →