TITLE=warpstore ROUTE=/engines/warpstore LOCALE=en MODIFIED=1792417593 URL=https://wiki.teletypegames.org/engines/warpstore L=1 #22 L=2 https://git.teletypegames.org/engines/warpstore L=3 #45 L=4 #47 L=5 #43 q│ A bstore engineq reads a 1WarpEngine catalog and writes some host's own game q│ library format. Only that last step differs per host. Talking to the API, q│ picking a release, matching the machine and remembering what went where is q│ the same work everywhere — cwarpstoreq is that work, in one file, shared by q│ every store engine. p ┌──────────────────────────────┐ p │ warpstore.py │ p │ catalog · releases · host │ p │ state · config · HTTP │ p └──────────────┬───────────────┘ p ┌──────────────┴───────────────┐ p ▼ ▼ p warp-engine-batocera-store warp-engine-retroarch-store p ROM folders + gamelist.xml .lpl playlists + thumbnails p │ │ p ▼ ▼ p EmulationStation RetroArch bRepository:0 2https://git.teletypegames.org/engines/warpstore bCatalog side: 1WarpEngine bUsers:0 3Batocera store engine · 4RetroArch store engine 1Why it exists 1═════════════ The Batocera store was 1073 lines, and only about a third was about Batocera. The RetroArch store needed the other two thirds, so the shared part moved out — with a second consumer already in hand, so the seam sits where the two engines really differ. Single file, Python 3 standard library only: the hosts range from a Batocera box (cpython30, no cpip0) to a desktop, and neither may be asked to install anything. An engine's installer drops cwarpstore.py0 next to the adapter script, and the adapter does cimport warpstore0. 1What an adapter supplies 1════════════════════════ 1. cDEFAULT_CONFIG0 — where things live on this host, which catalog platform maps to what. 2. caccept(spec, sw)0 — which catalog entries this host can run, and how they are grouped. 3. bThe code that writes the host's library format. p import warpstore as ws p p def accept(spec, sw): p """Return (scope, extras), or raise ws.Skip(reason) to reject the title.» p """ p if spec["system"] not in available_systems: p raise ws.Skip(f"no '{spec['system']}' ROM folder on this box") p return spec["system"], {"system": spec["system"]} p p cfg = ws.load_config(path, DEFAULT_CONFIG, required=("paths.subfolder",)) p ws.init(path) p games, skipped = ws.select_games(cfg, ws.fetch_catalog(cfg, use_cache=True),» p accept) cscope0 is how the host groups its library — a Batocera system, a RetroArch playlist. cstate.json0 is keyed by c:0, so the same software name in two scopes stays two entries. 1What is in it 1═════════════ t┌────────────┬───────────────────────────────────────────────────────────────┐ t│h Area t│h Functions t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 Logging t│0 cset_tag0, cset_verbose0, clog0, cdebug0, cdie0 t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 Files t│0 cload_json0, cwrite_json0, cwrite_atomic0, cwithin0, cprune_empty_dirs0 t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 Config t│0 cdeep_merge0, cload_config0 t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 Store home t│0 cinit0, then cHOME0, cCONFIG_PATH0, cSTATE_PATH0, cCATALOG_CACHE0 t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 HTTP t│0 chttp_get0, chttp_download0, capi_url0, cdownload_url0, cuser_agent0 t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 Host t│0 cmachine0, chost_os0, chost0, cresolve_for_host0 t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 Catalog t│0 cfetch_catalog0, cselect_games0, csurvey_catalog0, cpick_release0, t│ t│0 t│0 casset_basename0, cdownload_image0 t│ t├────────────┼───────────────────────────────────────────────────────────────┤ t│0 State t│0 cload_state0, csave_state0, cgame_key0, crecord_scope0, t│ t│0 t│0 crecords_by_scope0, cmatch_keys0, climit_to_names0 t│ t└────────────┴───────────────────────────────────────────────────────────────┘ Five of them carry decisions worth knowing. cselect_gamesb vs. csurvey_catalog0 — cselect_games0 returns what the store can install. csurvey_catalog0 returns that plus one record per title this machine cannot install, with a creason0 code (cplatform_off0, chost_asset0, cno_asset0, cvetoed0, and from 1.5.0 cgated0 — a title the host sells, which an anonymous store cannot download; the record then carries its cpurchase_url0) and a sentence. A client can grey those out instead of hiding them, which is what the WarpEngine Client does in its own port. Titles the store ichooses0 not to offer are in neither list — a wrong status or an exclude list is editorial, not a limit of the machine. cfetch_catalog0 asks the host to filter by cowner_id0 and ccatalog.statuses0 itself (cstatus=released,archived0), and follows cnextPage0 if the host pages the answer anyway — so a host-side page size can never quietly cut a sync short. Both only shorten the download: csurvey_catalog0 applies the same filters again, so a host that ignores a parameter changes nothing. cdownload_url0 points at cGET /api/download0 rather than c/file/0, so the host sees every download. Whether it counts them is the host's call: teletypegames.org deliberately leaves the cwarpstore/*0 user agent (cuser_agent0) out of its download counts (cignored_user_agents0), so installs by a store do not show up in the statistics. cresolve_for_host(value, host)0 — a config value that may depend on the machine. A plain string is the same everywhere, which is what a ccartridge0 is: data for an emulator. Otherwise the value is a map and the most specific key wins: p {"linux-aarch64": …, "aarch64": …, "linux": …, "*": …} No matching key and no c*0 gives cNone0, and the caller reports the title as skipped instead of installing something that cannot run. This generalises the Batocera engine's cresolve_for_arch0: Batocera only ever answered clinux0, but a RetroArch host may be Windows, macOS or Android — which decides what a libretro core file is called (c.dll0, c.dylib0, c_android.so0). cwithin(path, root)0 — guards every delete. A store may only remove files from the subtree it owns, never from the user's library and never from another store's. cprune_empty_dirs(dirs, root)0 — the uninstall side of the same rule. Only empty directories go, and only inside croot0: one stray file keeps a directory. This is what lets cpurge0 (and so cuninstall.sh0) leave the host as it was. cwrite_atomic0 — nothing is written in place. A store interrupted mid-sync would otherwise leave a half-written playlist or gamelist, which is worse than an old one. 1Taking every store off a machine 1════════════════════════════════ p curl -fsSL https://git.teletypegames.org/engines/warpstore/raw/branch/master» p /uninstall.sh | sh Use this to strip the whole framework without remembering what is installed: it finds every store home under the known roots, has each store's bown engine purge what it installed, then deletes the store, its launcher and the engine files. For a single store, prefer that engine's own cuninstall.sh0 — it knows the host extras, like a Batocera Ports entry. t┌─────────────────────────────────────┬──────────────────────────────────────┐ t│h Environment variable t│h Meaning t│ t├─────────────────────────────────────┼──────────────────────────────────────┤ t│0 cDRY_RUN0 t│0 c10 prints what would go and removes t│ t│0 t│0 nothing t│ t├─────────────────────────────────────┼──────────────────────────────────────┤ t│0 cKEEP_GAMES0 t│0 c10 removes only the scripts, keeps t│ t│0 t│0 the games t│ t├─────────────────────────────────────┼──────────────────────────────────────┤ t│0 cFORCE0 t│0 c10 purges even while RetroArch is t│ t│0 t│0 running t│ t├─────────────────────────────────────┼──────────────────────────────────────┤ t│0 cBATOCERA_STORE_ROOT0, t│0 where to look on a Batocera box t│ t│0 cBATOCERA_PORTS_DIR0 t│0 t│ t├─────────────────────────────────────┼──────────────────────────────────────┤ t│0 cSTORE_ROOT0, cBIN_DIR0 t│0 where to look on a desktop t│ t└─────────────────────────────────────┴──────────────────────────────────────┘ If an engine cannot finish (RetroArch running, ROMs root unmounted) the run stops there, rather than deleting the engine that knows what it installed. The script is POSIX csh0, not bash, so c| sh0 works where c/bin/sh0 is dash. The engines' own installers are too. 1State 1═════ cstate.json0, version 2, keyed c:0: p { p "version": 2, p "installed": { p "c64:blessingofra": { "name": "blessingofra", "scope": "c64", "asset": "» p blessingofra-2.0.0.prg", "…": "…" } p } p } crecord_scope()0 falls back to a record's csystem0 field, which is what the Batocera store wrote before this module existed. There the two were the same string, so an installed box keeps working with no migration and re-downloads nothing. A cversion: 10 file, keyed by bare software name, is re-keyed on first run. 1Versioning 1══════════ The engines fetch cwarpstore.py0 from wherever their installer's cWARPSTORE_RAW_BASE0 points — cmaster0 by default, so a change here reaches a store on its next install, which is also the reason to keep the surface small. A store repository may pin it instead: the 5TTG Batocera Store fetches both its engine and cwarpstore0 at fixed commits, and bumps them after a release. c--version0 on either engine prints both numbers, e.g. c3.1.0 (warpstore 1.2.0)0.