Wednesday, June 2, 2021

Spaced Repetition Flashcards

After two years using Anki, I decided to write my own spaced repetition flashcards software.

I was using the Anki v2 scheduler but it had various faults and limitations, the most serious of which were a fixed number of new cards per day, lack of API to implement an alternate scheduler, frequently changing add-on API and overly complex build environment that requires familiarity with several languages and build tools. 

I created add-ons that overcame some of the faults but it became too difficult to maintain them. And they didn't achieve all that I wanted. Because of how many of the scheduling features are implemented, many problematic aspects are difficult or practically impossible to alter or improve. It became evident that it would be less work and better outcome to write my own software than to continue struggling with Anki.

I use only a small subset of Anki features. Only an single instance of the desktop version (so no syncing) and only simple cards - no cloze cards. The Anki application was vastly more complicated than I needed.

So, I wrote my own. It is browser based with the server running on nodejs. It is simple: less than 1000 lines of JavaScript and a few templates. It's a bit crude yet, but I have been using it to study for over a week now. It presents all my cards as well as Anki did. It has, in my opinion, superior scheduling. The build tools are simple and well known - in other words: easy to install and use. I only need to know one programming language: JavaScript, or five if you include the SQL, Handlebars templates, HTML and CSS as languages. All but the Handlebars templates are ubiquitous and familiar, and Handlebars templates sufficient for this purpose, are simple.

About the same time, I updated my Anki add-on again, to support up to the then current 2.1.43 (and probably a few subsequent - until the internals change again). This is likely to be the last time I will update the add-on as I am not using it or Anki any more. 

Scheduling is simpler and more consistent than Anki. New cards are automatically regulated to maintain a steady workload of about 1 to 2 hours of study per day. The number of cards to be reviewed each day is somewhat random, but if study time in a day (actual or projected) exceeds 1 hour then no new cards are presented. This allows me to focus on the cards I am already learning, until I learn them well enough that cards to be studied / study time fall below the threshold, after which new cards are presented again. Other than the limit on total study time, there is no limit on new cards.

The scheduler prioritizes cards with shorter intervals over cards with longer intervals. This doesn't make much difference when you are up to date but it makes a big difference with a backlog, as I had after working on on this new app for a few days instead of studying. Even with a large backlog, the challenging cards don't get deferred by the backlog. It seems, in this way, to be much better than Anki. A few days after I got back to studying, I had cleared my backlog and was seeing more new cards than I had in a long time with Anki.

I have no research that proves my scheduler is more effective than Anki's but at least it doesn't have the misfeatures that I experienced with Anki: fixed number of new cards per day; excessive complexity of new, learning, re-learning and review cards and queues; a tendency to gross overload and dysfunctional review ordering, resulting in overall failure to learn; adverse interactions of new card limits between nested decks; lack of control of new card order; and limitation of the scheduling algorithm to units of days.

It is based on an import and modification of my old Anki database. I haven't yet written anything to import an Anki (or other) shared deck, but I probably will. However, currently I have about 30,000 new cards, so no need for more any time soon. I wrote an utility to import the Anki database but probably won't use it again: it would be too much of a setback to revert to what I was doing in Anki weeks ago and I don't study with Anki any more. I suppose I might import a new database if I used Anki to download some other decks and wanted to keep multiple databases, but it would probably be better to write an import of shared decks directly than to do it through Anki.

There is a very crude interface for editing notes: just the fields. So, I can correct errors as I find them. I'll add features to add and remove notes at some point.

New card ordering isn't lost when a card is studied. As a result, it is possible to reset a deck to it's original state and start over with the original ordering of new cards. This isn't possible with Anki because the initial ordering is in the due field of the cards, which is overwritten when the card is viewed.

Some of the serialized data in the Anki database is no longer used but there is more to be done. There is nothing fundamentally wrong about storing serialized data structures, but using a non-standard format is bad. I might revise some of the serializations to JSON. Alternatively, I might deserialize them into separate fields in the database. But, for the moment, I have cracked the rust serialization algorithm, so there is no immediate need.

There is room for improvement in the review log, but I can produce basic statistics such as counts of cards studied and due per day, study time per day, number of new cards viewed per day, etc.

Labels