Rahul Juliato: Taming Emacs Cache and Temporary Files

Wait 5 sec.

You open your favorite editor, and, out of the box, Emacs writes statefiles all over the place. It puts bookmarks in ~/.emacs.d/, drops#auto-saves# next to every file you edit, leaves .~lockfiles~ inyour projects, scribbles recentf, history, saveplace,projects, transient/history.el, tramp, network-security.data,multisession/, url/, image-dired/, erc/, rcirc/... you getthe picture. The features are great, don't get me wrong. The defaultlocations are what annoy me.Most of those files are not noise. They are the state that makesEmacs feel like Emacs across restarts. So you cannot just deletethem. But you also do not want them spread across N differentdirectories with cryptic names. (I know I don't.)I want one directory I control, every cache-bound variable pointinginside it, and a switch to flip it between "persistent inside myconfig" and "ephemeral in /tmp". No external package, no namingscheme to memorize, just a small pattern I can read top to bottom ininit.el.My emacs-solo config usedto hard-code a single relative path under user-emacs-directory forall these cache files. It worked, but it was inflexible. A useropened an issue asking if the cache location could be changed withoutforking the config, since they wanted those files somewhere elseentirely. So I redesigned it around a defcustom root and an alistof relative paths, switchable through M-x customize. The versionbelow is the same idea with the emacs-solo names dropped.Our goalPatch these three pieces:→ cache directory: holds every piece of mutable state Emacs writesduring a session.→ alist map: lists every variable that needs to be pointed at thatdirectory.→ helpers: one resolve a key to an absolute path, and anotherpre-creates every directory so we never get "no such file ordirectory" warnings at startup.In practice, a .emacs.d directory ends up looking like this:.├── cache│   ├── auto-saves│   ├── erc│   ├── image-dired│   ├── multisession│   ├── rcirc│   ├── transient│   ├── url│   └── ...├── eln-cache│   └── 32_0_50-25c5b284├── eshell│   └── history├── init.el└── tree-sitter├── libtree-sitter-markdown-inline.dylib└── libtree-sitter-markdown.dylibOur switchable base directoryStart with a defcustom for the root. Using defcustom (instead ofdefvar) lets you change the value through M-x customize withoutediting init files, and the choice persists in custom.el:(defcustom my/cache-directory (expand-file-name "cache/" user-emacs-directory) "Base directory for Emacs cache files.All entries in `my/cache-paths' are resolved relative to thisdirectory. Choose one of the presets or supply any custom path.Changes take effect after restarting Emacs." :type `(choice (const :tag "Inside Emacs config (cache/ in user-emacs-directory)" ,(expand-file-name "cache/" user-emacs-directory)) (const :tag "System temp (/tmp/emacs-cache/)" "/tmp/emacs-cache/") (directory :tag "Custom directory")) :group 'my)The default puts everything under ~/.emacs.d/cache/, which keepsthe rest of ~/.emacs.d/ clean and lets you back up or delete thewhole cache as one directory. The /tmp/ preset is for sessions thatshould leave no trace. The directory choice lets you point at anycustom path.I use user-emacs-directory instead of a literal ~/.emacs.d/because Emacs 29 added the --init-directory flag, which lets youlaunch Emacs against any config directory. I use it constantly to tryother people's configs side by side with my own without touching~/.emacs.d/. Anchoring on user-emacs-directory means the cachefollows whichever config Emacs was started with, instead of everyalternate config writing into the default ~/.emacs.d/ and steppingon each other.In my early-init.el I already load custom-file, but I reload ithere too. customize writes its changes to custom.el immediately,so if you change the cache directory mid-session through M-x customize and then restart, you want the new value to apply beforethe rest of init.el runs and calls my/cache--path:(when custom-file (load custom-file 'noerror 'nomessage))The when guard is there because in a brand-new config custom-fileis nil, and (load nil 'noerror 'nomessage) signals(wrong-type-argument stringp nil) and aborts the rest ofinit.el. The 'noerror flag suppresses file-error, not typeerrors.This whole reload is a bit of a hack. Loading custom-file twiceduring startup just to get a defcustom value hot before the nextform is not what Emacs intends. It only matters the first time youswitch presets. Without it, you would wonder why your new/tmp/emacs-cache/ is empty while your old ~/.emacs.d/cache/ keepsreceiving writes.Our single alist as the source of truthThe next piece is the mapping from keys to relative paths.How you organize this alist is your call. The list below is what Iwire through my/cache-path in my own config, but plenty of userswill split things differently. I deliberately keep tree-sitter/,eln-cache/, and eshell/ outside this alist. tree-sitter andeln-cache are populated by long-running, expensive processes(grammar compilation, native compilation) that I want to survive whenI reset my cache, so they live next to the config underuser-emacs-directory directly. eshell carries my command historyand aliases, which I treat more like dotfiles than throwaway state.Your workflow will pull these differently, and that is fine.(defvar my/cache-paths '(;; Files:(bookmark-file . "bookmarks")(ielm-history-file-name . "ielm-history.eld")(project-list-file . "projects")(recentf-save-file . "recentf")(savehist-file . "history")(save-place-file . "saveplace")(transient-history-file . "transient/history.el")(transient-levels-file . "transient/levels.el")(transient-values-file . "transient/values.el")(tramp-persistency-file-name . "tramp")(nsm-settings-file . "network-security.data");; Directories:(auto-saves . "auto-saves/")(auto-saves-sessions . "auto-saves/sessions/")(multisession-directory . "multisession/")(url-configuration-directory . "url/")(image-dired-dir . "image-dired/")(erc-log-channels-directory . "erc/logs/")(erc-image-cache-directory . "erc/images/")(rcirc-log-directory . "rcirc/logs/")) "Alist of (KEY . RELATIVE-PATH) for cache locations.RELATIVE-PATH is resolved against `my/cache-directory'.A trailing slash on RELATIVE-PATH marks the entry as a directory.")The convention I follow: keys are usually the names of the Emacsvariables they will fill. That keeps grep useful for both thedefinition and the usage.A trailing slash on the value marks a directory (this matchesdirectory-name-p in Emacs). No trailing slash means a file. Thenext helper uses that distinction.Our helpersThe first helper turns a key into an absolute path:(defun my/cache--path (key) "Return the absolute path for KEY in `my/cache-paths'." (let ((rel (cdr (assq key my/cache-paths))))(unless rel (error "my/cache--path: Unknown key %S" key))(expand-file-name rel my/cache-directory)))assq does an eq lookup on symbol keys. The unless rel branch isthere because typos in :custom blocks are silent otherwise: youwould get nil and Emacs would happily write to nil, which failsin confusing ways. Better to error at config load.The second helper pre-creates every directory the alist mentions, sowe never get "directory does not exist" warnings from packages onfirst run:(defun my/cache--ensure-dirs () "Create every directory referenced by `my/cache-paths'.Entries ending in `/' are created directly; other entries have theirparent directory created." (dolist (entry my/cache-paths)(let* ((abs (my/cache--path (car entry))) (dir (if (directory-name-p abs)abs (file-name-directory abs)))) (make-directory dir t))))(my/cache--ensure-dirs)If the entry is a directory (directory-name-p returns t forvalues ending in /), it is created as-is. If the entry is a file,only the file's parent directory is created.The t argument to make-directory is the "parents" flag, theequivalent of mkdir -p. So transient/history.el correctly creates/transient/ even though we never list it explicitly.Wiring it into use-package :customNow every cache-bound variable points at our helper. Inside ause-package emacs block (or wherever you set built-ins):(use-package emacs :ensure nil :custom (bookmark-file (my/cache--path 'bookmark-file)) (ielm-history-file-name (my/cache--path 'ielm-history-file-name)) (project-list-file (my/cache--path 'project-list-file)) (recentf-save-file (my/cache--path 'recentf-save-file)) (savehist-file (my/cache--path 'savehist-file)) (save-place-file (my/cache--path 'save-place-file)) (transient-history-file (my/cache--path 'transient-history-file)) (transient-levels-file (my/cache--path 'transient-levels-file)) (transient-values-file (my/cache--path 'transient-values-file)) (nsm-settings-file (my/cache--path 'nsm-settings-file)) (multisession-directory (my/cache--path 'multisession-directory)) (url-configuration-directory (my/cache--path 'url-configuration-directory)) ;; The two below are *not* backups; they keep auto-save state ;; without scattering `#file#' next to every edited buffer. (create-lockfiles nil) ; no `.#file' lock files at all (make-backup-files nil) ; no `file~' tilde backups at all (auto-save-default t)) ; auto-save *is* kept, just redirected belowOn the three settings at the bottom:→ create-lockfiles nil and make-backup-files nil are personaltaste. Lockfiles warn another Emacs not to clobber your buffer, butthey break frontend tooling that watches directories for changes(TypeScript, Vite, esbuild). The tilde backups I find redundantwith modern VCS and auto-save.→ auto-save-default t stays on because auto-save is whatrescues you when Emacs crashes. We just want it somewhere else.Redirecting auto-savesAuto-save needs two extra settings because Emacs uses path transformsfor it instead of single-value variables:;; We want auto-save, but no #file# cluttering, so everything goes;; under our cache/ tree. Directories are pre-created.(setq auto-save-list-file-prefix (my/cache--path 'auto-saves-sessions) auto-save-file-name-transforms `((".*" ,(my/cache--path 'auto-saves) t)))auto-save-list-file-prefix controls where the "list of files withpending auto-saves" lives. This is what M-x recover-session reads.Pointing it at our auto-saves/sessions/ directory means you canstill recover after a crash, without ~/.emacs.d/auto-save-list/cluttering your config.auto-save-file-name-transforms is a list of (REGEX REPLACEMENT UNIQUIFY) triples. The t at the end is the uniquify flag, whichencodes the original path into the filename so two files with thesame name in different directories do not collide in the auto-savefolder.Both end up in /auto-saves/.TRAMP, viper, and other late bindingsSome variables are not safely set in :custom, because they aredefined inside packages that load later. For those, use setopt(which respects defcustom setters) inside the :config block:(setopt tramp-persistency-file-name (my/cache--path 'tramp-persistency-file-name))(setopt viper-custom-file-name (my/cache--path 'viper-custom-file-name))setopt is the modern equivalent of setq for customizablevariables. It runs any :set function the variable defines, whichtramp-persistency-file-name does (it triggers a reload). Plainsetq would skip that and leave TRAMP confused.What you get→ ~/.emacs.d/cache/ (or wherever you point it) containseverything. du -sh tells you how much state Emacs is hoarding.rm -rf resets it all without touching your config.→ M-x customize-variable RET my/cache-directory lets you flipbetween persistent and ephemeral modes without editing initfiles. Useful for screencasts where you want zero history showing,or for testing whether a problem is config or state.→ New packages are a two-line change. When you adopt, say,newsticker, you add one entry to my/cache-paths and one(my/cache--path 'newsticker-dir) line in the package's:custom. The directory is auto-created on next restart.→ You can grep for it. grep my/cache--path init.el lists everyplace Emacs writes state, and the alist tells you what kind.The complete codeHere is everything in one block. Copy this into some temporary folderinside the init.el file. After that, cd into this temp directoryand run emacs --init-directory=./. You can them navigate files, useEmacs features and check where the created files end up to.(defcustom my/cache-directory (expand-file-name "cache/" user-emacs-directory) "Base directory for Emacs cache files." :type `(choice (const :tag "Inside Emacs config (cache/ in user-emacs-directory)" ,(expand-file-name "cache/" user-emacs-directory)) (const :tag "System temp (/tmp/emacs-cache/)" "/tmp/emacs-cache/") (directory :tag "Custom directory")) :group 'my)(when custom-file (load custom-file 'noerror 'nomessage))(defvar my/cache-paths '(;; Files:(bookmark-file . "bookmarks")(ielm-history-file-name . "ielm-history.eld")(project-list-file . "projects")(recentf-save-file . "recentf")(savehist-file . "history")(save-place-file . "saveplace")(transient-history-file . "transient/history.el")(transient-levels-file . "transient/levels.el")(transient-values-file . "transient/values.el")(tramp-persistency-file-name . "tramp")(nsm-settings-file . "network-security.data");; Directories:(auto-saves . "auto-saves/")(auto-saves-sessions . "auto-saves/sessions/")(multisession-directory . "multisession/")(url-configuration-directory . "url/")(image-dired-dir . "image-dired/")(erc-log-channels-directory . "erc/logs/")(erc-image-cache-directory . "erc/images/")(rcirc-log-directory . "rcirc/logs/")) "Alist of (KEY . RELATIVE-PATH) for cache locations.")(defun my/cache--path (key) "Return the absolute path for KEY in `my/cache-paths'." (let ((rel (cdr (assq key my/cache-paths))))(unless rel (error "my/cache--path: Unknown key %S" key))(expand-file-name rel my/cache-directory)))(defun my/cache--ensure-dirs () "Create every directory referenced by `my/cache-paths'." (dolist (entry my/cache-paths)(let* ((abs (my/cache--path (car entry))) (dir (if (directory-name-p abs)abs (file-name-directory abs)))) (make-directory dir t))))(my/cache--ensure-dirs)(use-package emacs :ensure nil :custom (bookmark-file (my/cache--path 'bookmark-file)) (ielm-history-file-name (my/cache--path 'ielm-history-file-name)) (project-list-file (my/cache--path 'project-list-file)) (recentf-save-file (my/cache--path 'recentf-save-file)) (savehist-file (my/cache--path 'savehist-file)) (save-place-file (my/cache--path 'save-place-file)) (transient-history-file (my/cache--path 'transient-history-file)) (transient-levels-file (my/cache--path 'transient-levels-file)) (transient-values-file (my/cache--path 'transient-values-file)) (nsm-settings-file (my/cache--path 'nsm-settings-file)) (multisession-directory (my/cache--path 'multisession-directory)) (url-configuration-directory (my/cache--path 'url-configuration-directory)) (create-lockfiles nil) (make-backup-files nil) (auto-save-default t) :config (setq auto-save-list-file-prefix (my/cache--path 'auto-saves-sessions)auto-save-file-name-transforms`((".*" ,(my/cache--path 'auto-saves) t))) (setopt tramp-persistency-file-name (my/cache--path 'tramp-persistency-file-name)))Adapt the alist to the packages you actually use.Other ResourcesIf you want to read more on the topic:→ https://www.gnu.org/software/emacs/manual/html_node/emacs/Auto-Save-Files.html→ https://www.gnu.org/software/emacs/manual/html_node/elisp/Variable-Definitions.html#index-defcustomIf you would rather install a package than maintain your own alist,the no-litteringpackage is a popular ready-made alternative that covers a wide set ofvariables out of the box.The version of this code I actually run, alist entries and all, livesin my emacs-solo configunder the CACHE PATHS heading of init.el. If you end up adaptingthis for your own setup, I would love to hear which variables youadded that I forgot.