I've been trying to upgrade from 10.10 to 10.11 for a while now, as the Android TV app keeps nagging me, and every attempt ended with impossibly long library scan times.

After some thorough investigation, it appeared that a Home Videos type collection causes unending scan (yet to be solved), but also that Jellyfin does a lot of writes to the config directory (either database or metadata or both). Mine's on spinning media part of a ZFS pool. I tried a few performance tuning options, such as testing the config dir with recordsize (similar to block size) of 4K, 8K, 64K, 128K and library scans fell from 30-40 minutes down to 8-13min with 4K-64K. The ZFS tuning wiki suggest 64K recordsize with LZ4 compression for SQLite workloads such as Jellyfin. That seems to work as well as 4K and 8K but likely is faster when reading thumbnails and such.

Note that upgrading to 12-rc3, which is supposed to speed up library scans did not improve scan times for me. Optimizing config/database write speed did. I cross-checked the culprit by experimenting with moving the config dir to NVMe and RAM. Both of those got the scan times down to 8-9 minutes compared to the optimized spinning media's 12-13.

So if you had upgraded (or about to) to 10.11 your library scans are (about to get) dog slow and your Jellyfin's config dir resides on spinning media, optimize its write performance for SQLite.

top 50 comments

sorted by: hot top controversial new old
[–] 148 points 2 months ago* (75 children)

Isn’t it ironic that you post this meme along with complex instructions to fixing a problem in jellyfin?

  • source
  • hideshow 75 child comments
  • [–] [S] 64 points 2 months ago* (last edited 2 months ago) (19 children)

    With great power come great .. uuh .. :D

    Also the meme is just bait so you read abt the problem and see if you need to solve it for your setup.

  • source
  • parent
  • hideshow 19 child comments
  • [–] 2 points 2 months ago (14 children)

    Yeah but tell me about this tool for Plex remote streaming...

  • source
  • parent
  • hideshow 14 child comments
  • load more comments (1 reply)
  • load more comments (55 replies)
    [–] 19 points 2 months ago (1 child)

    I run the DB on the SSD and media on the hdds and haven't had any issues, but I don't have a huge library though. However, things are generally speedy. I thought the higher record sizes were recommended for media, not the DB? but if it worked for you, that's good

  • source
  • hideshow 1 child comment
  • [–] [S] 3 points 2 months ago

    64K is generally small recordsize. For media they recommend 1M. My db was sitting on the default 128K recordsize and perforned fine on 10.10. It doesn't matter much for reads because ZFS keeps it in RAM and if it gets evicted it goes into SSD cache (l2arc). But for writes the cache wouldn't help unless I disable sync which risks data loss. I don't think 10.10 did much writes during lib scan so write perf only mattered when something changed.

  • source
  • parent
  • [–] 12 points 2 months ago* (1 child)

    If you use folders a lot, the library will get faster with v12 https://github.com/jellyfin/jellyfin/issues/15141

    OP said this didn’t improve their situation though.

  • source
  • hideshow 1 child comment
  • [–] [S] 2 points 2 months ago*

    Not sure if this is the problem I was hitting with my Home Videos library type. It's got directories with YouTube videos. I tried re-adding it in 12-rc3, which was released a month after the nightly mentioned in the issue thread. Still couldn't complete. I ended up re-adding the media as Shows library. Works okay with the directory structure I have.

    The write performance was recorded without the problematic library (deleted).

  • source
  • parent
  • [–] 6 points 2 months ago (4 children)

    How large is the database? Because there is a way to fully load SQLite into memory, which would speed everything up drastically for a scan. At the end of the scan, you could just write back everything to the disk, then it is just one blob of sequential write, which is fast enough on spinning rust.

  • source
  • hideshow 4 child comments
  • load more comments (3 replies)
    [–] 3 points 2 months ago (2 children)

    I've had initial scans take hours and scans afterwards usually also take some minutes. Why do you think that is an unreasonable amount of time to check all entries for updated metadata?

    unending != 40 minutes.

  • source
  • hideshow 2 child comments
  • [–] [S] 3 points 2 months ago (1 child)

    That would be reasonable. I did repeated rescans and only counted subsequent rescans. For me the initial scan after upgrade took a bit more but not hours. Subsequent scans took less. E.g. 20min -> 13min for write-optimized filesystem. So that's reasonable, although 10.10 was way faster. Library scans are expected to get faster in 13 according to some Github threads I read.

    When I had the broken Home Videos library I waited 3 days for the initial scan to complete and it did not. Repeated rescans did not seem to complete although I didn't wait 3 days for them. I'm not taking into account those scan times. Something was wrong with this library type on 10.11 and/or my media. Worked fine on 10.10.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 2 months ago* (1 child)

    Hey not bad for the developer...he vibe coded it right

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 2 points 2 months ago

    Thanks, will take a look a my setup

  • source
  • [–] 2 points 2 months ago (1 child)

    I've always kept my apps and databases on solid state media, putting them on spinning rust, even in a well tuned ZFS pool with lots of disks is way too slow for their (admittedly inefficient) database.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    load more comments
    view more: next ›