Anki Backlog Hell

 People have written about Anki Ease Hell:

• Ease Hell in Anki
• Ease Hell in Anki
• Anki Design Study: Adverb Phrases
• Auto Ease Factor

But there is another kind of hell with Anki, which I call Backlog Hell.

Backlog Hell results from two features of Anki:

1. Cards are presented in the order they are due
2. New cards are presented regardless of backlog

When you have a backlog of due cards, you do not see them promptly when they are due. This isn't an issue with a small backlog but becomes an issue with a larger backlog.

Consider what happens if you are reviewing about 1,000 cards per day and you take a break from study for a week. You may take a break because you have exams, vacation, work or illness. It doesn't matter why. When you return, how many cards do you have due? After one week, you will have about 7,000 cards due for review. Maybe a bit less: some of those cards are short interval.

You normally are reviewing 1,000 cards per day.  Maybe this takes you 30 minutes per day (about 2 seconds per review if they are quick) or maybe 2 hours per day (about 8 seconds per review if you take a bit more time on each card).

To get through your backlog, you might be able to double your study time. If you do, you can review 2,000 cards per day instead of your usual 1,000 cards. You should clear your backlog quickly, right? In one week, you should have cleared your backlog, right?.

Wrong! it will take much more than 1 week to clear your backlog. It might never happen. Instead, your backlog might get larger every day.

If you know your cards well (i.e. they are almost always Good or Easy, even if you don't see them promptly) then you will clear your backlog.

If a card has an interval of 100 days and you see it after 107 days, it doesn't make much difference. If it was going to be Good at 100 days, it will probably be Good at 107 days. You will clear these from your backlog. Even if you don't get to it the first day it is due. Seeing them after 114 or 120 days isn't too much different from 100 days. Not optimum, but often you will still remember.

But what about the cards that you were just learning: the majority of your daily review? What happens to a card that you should have reviewed after 7 days? You will not see it again for at least 14 days, maybe longer. Will it still be Good after two or three weeks?

You end up clicking Again. You should see these cards again soon. But you still have a backlog, so you don't see it until you have seen all the other cards in your backlog. Maybe not for a few days or weeks.

You continue reviewing cards long after they were due. Can you recall them, long after they were due? Sure, sometimes you will. But if the scheduling algorithm is good, seeing them long after they were due will not be as effective as seeing them when they were due: the whole point of the scheduler is to present them at the optimum interval: shorter or longer intervals are sub-optimum.

So, just when you need to optimize your study: to clear the backlog as quickly as possible - Anki goes into a sub-optimal mode.

If your backlog is too big, it will be so sub-optimal that you can't catch up.

And Anki adds new cards every day. What happens to a new card added today? You should see it again in a few minutes, or maybe tomorrow. But when do you see it?

Because Anki presents cards according to due date and all the cards in your backlog were due before today, you will not see that new card again until you have reviewed all the cards in your backlog. If you have 7,000 cards in your backlog and you can review 2,000 per day, that's over 3 days until you can see that card again.

Can you learn your new cards if you only see them once every 3 days? How about once a week? Or once a month? If I wasn't doing anything else, maybe I could. But when I am reviewing 2,000 cards per day, I can't. It comes up again in a few days but I should have seen it again in 10 minutes. I can't remember it. So I click Again. It is due in 10 minutes but there are still 7,000 cards that I have to review first, before I will see it again. And I can't review 7,000 cards in 10 minutes, so I see it late again: days late.

And still Anki makes the problem worse by adding more new cards every day.

Seeing cards significantly after their due date, your learning will be sub-optimal.  With a small backlog: no problem. With a large backlog: you will never catch up. You will study more and more but see your backlog get a bit bigger each day. Remember: you were studying 1,000 cards per day. Some of these were short interval cards but some were long interval cards: there will be more of these added to your backlog each day. 

Eventually, you will stop learning almost completely: forever reviewing but never progressing: a Sisyphean torment: Backlog Hell.

Every time you see a card that you can't remember, because you haven't seen it in too long and you are overloaded, you click Again. Doing that a few times sends you into Ease Hell.

Ease Hell is bad enough, but combine it with Backlog Hell and you are doomed to failure. Your backlog will grow. You will never catch up. You will spend more and more time to learn less and less. When you don't see your cards on time: when you see your new cards days late, you will not learn effectively.

How to escape Backlog Hell?

First, manually reduce your new cards per day to 0. There is no point adding to the problem. You can start learning new cards again after you clear your backlog.

Then, study only a subset of cards each day. You can study select decks or create custom decks. Your objective should be to review cards on time. You need to optimize your learning to get through the backlog.

It is better to review some cards on time and learn them than it is to review all your cards (or as many of them as you can review in a day) days or weeks late and not learn any of them.

With no new cards being added, if you continue learning effectively you should eventually clear your backlog.

Alternatively, you can set all the cards in your backlog back to new. Then they will be re-introduced gradually, according to your new card limit.

But, do you really want to set 7,000 cards back to new? That's tremendous setback: another torment.

I ended up giving up on Anki. I didn't want to reset all my cards to new or other manual workarounds, and I didn't want to forever be in Ease Hell and Backlog Hell. I wanted to learn and I wanted to spend my time learning, not fussing with Anki. I couldn't fix the Anki scheduler so I wrote srf. I still take occasional breaks from study and have backlogs. It still takes time to clear a backlog, but I make reliable progress and I don't have to fuss with the scheduler: it just works. 

On the other hand, the srf scheduler is easy to change if I want to: there are parameters for minor adjustments and it is just some JavaScript and SQL queries, if I want to make more fundamental changes. I have changed it several times since first writing srf. It is quite different from Anki in its details, though similar in its principles and objectives. It is now settled to something I am quite comfortable with and a vast improvement over the Anki scheduler.

Testing Express based apps

All of the examples of automated testing Express based apps that I have found use supertest. But supertest is based on superagent and I don't like the API of superagent and, therefore, supertest.

But it is not necessary to use supertest. All it does is run an instance of the app and submit queries to it, then muddle assertions with the HTTP client. It is easy, more flexible and cleaner to run the app directly then use whatever means to submit requests and assert assertions: hopefully from independent packages.

For example:

'use strict';

const t = require('tape');
const app = require('../app');
const http = require('http');
const axios = require('axios');

const server = http.createServer(app);
server.listen(0, '127.0.0.1');
server.on('listening', () => {
  const addr = server.address();
  const address = addr.address;
  const port = addr.port;
  console.log('addr: ', addr);
  console.log('listening on ' + address + ':' + port);
  runTests(port)
  .then(() => {
    server.close();
  })
  .catch(err => {
    console.log('tests failed with: ', err);
  });
});

function runTests (port) {
  return new Promise((resolve, reject) => {
    const request = axios.create({
      baseURL: 'http://localhost:' + port + '/'
    });

    t.test('GET / returns 200', (t) => {
      request({
        url: '/',
      })
      .then(res => {
        t.equal(res.status, 200, 'Status is 200');
      })
      .catch(err => {
        t.fail(err);
      })
      .finally(() => {
        t.end();
      });
    });
 

    t.onFinish(() => {
      resolve();
    });

    t.onFailure(() => {
      reject(new Error('something went wrong'));
    });
  });
}

This is simple enough: almost as simple as using supertest, and it has advantages:

• Use any http client code you like
• Use any assertion library you like

 

Recording audio that is playing

I sometimes want to record what is playing on my system. Not from the microphone, but what is playing to the headphones or speakers.

I am running Debian  Bullseye. It has a particular configuration of ALSA and PulseAudio that makes it easy.

The command 'pactl list' lists pulse audio devices, including all the sources.

On my system, one of those sources has the name 'alsa_output.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp__sink.monitor' with description 'Monitor of Tiger Lake-LP Smart Sound Technology Audio Controller Speaker + Headphones'. I see this description on the mixer Input Devices tab and it seems to correspond to what is playing on the headphones or speakers.

 arecord -f cd -t wav --max-file-time 3600 --device=pulse:alsa_output.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__hw_sofhdadsp__sink.monitor mon.wav

I suspect that the form of device name (`pulse:xxx`) depends on the configuration in /etc/alsa/conf.d/50-pulseaudio.conf:

# Add a specific named PulseAudio pcm and ctl (typically useful for testing)

pcm.pulse {
    @args [ DEVICE ]
    @args.DEVICE {
        type string
        default ""
    }
    type pulse
    device $DEVICE
    hint {
        show {
            @func refer
            name defaults.namehint.basic
        }
        description "PulseAudio Sound Server"
    }
}

ctl.pulse {
    @args [ DEVICE ]
    @args.DEVICE {
        type string
        default ""
    }
    type pulse
    device $DEVICE
}

I think this takes the xxx from 'pulse:xxx' as a parameter and allows access to the PulseAudio device by name.

Thunderbird keyboard shortcut customization

I am running Thunderbird 91.4.

I don't like the default Ctrl-V pasting formatting.

I installed the tbkeys-lite add-on and searched for two hours for any examples of configuration of the compose section. Didn't find a single example, and very few examples for the main section.

Issue 80 asked for examples and there was 1 reply.

http://kv5r.com/computers/thunderbird-paste-without-formatting/

Thunderbird Paste Without Formatting looks like it's relevant, but it is for an old version of Thunderbird - the files it references don't exist in my installation.

Finally, I tried the following and, fortunately, it worked:

{
  "ctrl+v": "cmd:cmd_pasteNoFormatting",
  "ctrl+shift+v": "cmd:cmd_paste"
}

With this, Ctrl-V pastes without formatting and Ctrl-Shift-V pastes with formatting.

npm install -g

As of about npm v5.0.0, the `npm install` command no longer installs from a directory. It is now an alias for `npm link`. This is fine in many cases, but `npm link` already existed and sometimes what one wants to do is install: not link.

For example, if a locally developed package is being installed globally so that all users can run a command it provides, it is generally best to install the package globally rather than link it, so that it is independent of the directory it was installed from.

Fortunately, it is not too difficult to overcome the elimination of this install option:

• cd <package-directory>
• npm pack
• npm install -g <package>-<version>.tgz

This works because when installing from a tgz file, `npm install` still installs rather than linking.

Worthless Guarantees from AliExpress and Netac Official Store

 I purchased an NVME SSD from Netac Official Store via AliExpress.

It is offered with "75-Day Buyer Protection Money Back Guarantee":

We promise your money back if the item you received is not as described, or if your item is not delivered within the Buyer Protection period. You can get a refund 15 days after the claim process finishes. This guarantee is in addition to and does not limit your statutory rights as a consumer, as granted by all mandatory laws and regulations applicable in your country of residence.

The package was delivered via postal service.

My local post office delivered the package to the wrong address and updated status to delivered.

I contacted my local post office and they confirmed that they delivered the package to the wrong address. They attempted to recover the package and deliver it to me but were unable to do so. They deemed the package to be lost.

My local post office cannot compensate me for the loss but they are willing to pay compensation for the loss if the sender makes a claim through the origin post office.

But AliExpress and the Netac Offical Store do not accept the emails I have forwarded from my post office confirming that they lost the package and did not deliver it to me.

They deem the communications from my local post office: "The evidence from NZPOST is not so clear to show the problems, please provide more clear evidence as required so that we can tell the problems obviously"

This is what NZ Post said:

Thank you for your patience. 

Sadly, your parcel has been delivered incorrectly, despite our attempts to recover this parcel from the incorrect address we have been unsuccessful. We have deemed this parcel to be missing. 

We recommend you contact the sender and request they initiate an enquiry/claim with the postal authority of the sending country. 

Please note claims for compensation for the loss or return of an international item can only be made by the sender from the country of origin. New Zealand Post has no provision for compensation for incoming international items. 

Our sincere apologies for the inconvenience caused by this incident.

How is this not clear? They made a mistake. They admit it. They are willing to pay compensation through the normal channels for international shipping.

International shipping is fraught with human error and acts of god that cause losses. Ships sink. Shipping containers go overboard. Warehouses go up in flames. All sorts of things. 

I don't mind that there was an error.

But obtuse refusal to accept the clear acceptance of fault and liability, to obtain the compensation offered and to send a replacement or refund is offensive.

AliExpress' and Netac Official Store's guarantees are worth nothing because they are in China and the guarantees are unenforceable from New Zealand, and neither AliExpress nor Netac Official Store are willing to honour them.

Update: NZ Post has changed their story. Now they say the package was scanned "near" my address and they don't know if it was delivered to my address and stolen or delivered to some other address. They maintain that in either case the package was "delivered" so they refuse to change the status in their tracking system. Still AliExpress is at fault for the delivery failure: they sent a valuable package with no signature required. Had a signature been required, the package would not have been left where it could be stolen. I would have gone in to pick it up and sign for it.

AliExpress has my money and I have nothing. Because they were negligent in how they sent it, not requiring a signature, allowing it to go missing rather than me receiving it. But from their perspective, all they have to say is: "The evidence from NZPOST is not so clear to show the problems,please provide more clear evidence as required so that we can tell the problems obviously"

The problem is: they have my money and I have nothing. But they don't care about that.

Limitations of Calibre and Calibre-Web as an epub book viewer

When viewing an epub book using Calibre, one's position in the book is indicated as percent, in the bottom right corner of the reader window. This is OK for a small book but it does not show fractions of a percent and therefore is the same for many pages of a longer book. It does have the virtue that it is independent of the size of the reader window. Calibre maintains a constant position when the window is resized: at least some of the text is common between the larger and smaller window.

The Calibre-Web reading window is worse: it doesn't indicate position in the book at all. This is made worse by the fact that if one resizes the reader window the reader jumps to a different position in the book. If one alternates between a small and large window, just flipping back and forth, position gradually moves to the beginning of the current chapter. There is no practical way to get back to the original position except to go back to the start of the chapter then scan forward to the desired location. This is very poor UX.

In the case of Calibre-Web, the behaviour is a result of using epubjs to display the book. I have made a small test app that uses default configuration and it exhibits the same faults: no indication of position and position jumps when window is resized.

The epubjs package has many options. There may be options to display position and to maintain position when window is resized, but I haven't found them yet. I tried capturing and repositioning after resize, but it is a bit of a nightmare of rapid-fire events and deciding when to save and when to restore position. There does not appear to be any option to disable the automatic redisplay on window size change and there is essentially no documentation of the implementation.

As with so much software these days, the documentation of epubjs provides the syntax of the API but very little to nothing regarding the semantics.

There are some clues about getting page information in issue 744. None of this defines what is meant by "page" but most of the comments imply that what is meant is: what is presented in the reader window.

Calibre-Web has an inbuilt table-of contents for epub books but if it is opened the right side of the content goes off-screen with no scrollbar. It is impossible to read the text with the table of contents open.

Calibre-Web allows bookmarks to be saved but the presentation of them is as a cryptic string like "epubcfi(/6/8[id_4]!/4/2/1:0)". This is a standard epub cfi but it is not human friendly. Given a list of these, how would a normal person know which refers to what? There is no way to annotate them. In contrast, a bookmark in Firefox can be edited to change the title, icon, etc.

The bookmarks persist between browser sessions.  And they persist through clearing cookies and site data. This suggests that they are stored on the server. But presumably not in the book itself as each user could have different bookmarks.

It appears that Calibre-Web has its own database, in addition to the Calibre database it accesses. Default on Linux is ~/.calibre-web/app.db. This includes table 'bookmark' with fields user_id, book_id, format and bookmark_key, the latter two beeing epub and an epubcfi string. So, bookmarks are per user, per book and persistent across devices and sessions.

Both epub2 and epub3 have support for page lists but various posts suggest that almost no epub2 and few epub3 readers actually use them.

The navigation file provides some guidance on navigation for epub3.

A book might be published in hard copy, with a particular layout of pages. The same book in epub format will break the text into different sections, depending on font, window size, etc. One might be interested in what page of the original hard copy publication is being viewed, regardless of how many reader windows it takes to view the complete page, or one might be interested in pages as determined by the reader window: one reader window full is one page.

Is it possible to position the reader window at an arbitrary position in the text? If so, then what 'page' is the reader at when the displayed text starts at the second character of the book? Or the third? etc.

If the definition of "page" depends on the size of the reader window, font, screen resolution for displaying images, etc. then page number is only relevant in the current reader window context. If one reads the same book on a different system, with a different reader size, with a different font size, etc. then the page numbers will all be different. "Page 237" will contain different text depending on all these (and probably other) factors.

Counting characters might be more consistent: current display starts at character "1073648 of 27634287", for example. But this isn't very human friendly.

Pages are a familiar concept in paper books, but what do they mean in an e-book?

How does one return to the same position in the book, when one re-opens it?

How does one refer to a part of the book in computer friendly terms (where character offset might be fine) and human friendly terms?

Is it possible to make a fixed page list that is independent of the reader window size, screen resolution, font size, etc.? 

There is a good discussion of epub3 page lists at epubsecrets. This includes use cases where consistency across different media formats, independent of the individual reader details, is useful.

The epub3 spec includes page-list nav element.

epub3 has support for fixed layout documents, with the introduction pointing out that by default epub3 documents are intended to adapt to the reader with reflow, etc.