Multi-Machine Magazine: a convention for networked filesystem schema

This was a sort of FHS-style dir convention I worked on a while back (2011), and have been using on a RHEL machine for a couple of years now.  Basically I wanted a place to put things might be shared in common by multiple VMs or hypervisors (like VM images).  Nowadays, I'd leave the distro's normal fs alone and use something like the OpenStack images service to worry about where these things go.  Also, other work has been done to make parts of the fs read-only to make things more instance-friendly.  So this whole idea is kind of obsolete.  It was an attempt to hierarchize that which cannot be done so.

So nobody should ever actually try what this draft says.  But I'm putting this up now (2014) in case it's ever interesting to anyone and catalyzes thought.  ...Which is unlikely.

There are some annoyances that can pop up, like how moving lots of important stuff like VM dirs can burn you when SELinux is running and mmm was created with a poor context.  But that's my fault, not SELinux's.  SELinux is great.

mmm

• mmm - Multi-Machine Magazine
  • A convention that may promote more useful network sharing of non-host-specific files.
  • Copyright and License: This document is licensed under the GNU FDL 1.3.
  • Abstract:
    • A personal convention is described that facilitates sharing of files among networked computers via migration of these files to designated physical volume and application of a novel path schema. A logically defined inversion of the migrated path allows allows utility of application or VM host commonality to emerge at the top level of a hierarchical directory tree. This is applied to an example schema for the hierarchical presentation of files and directories in the Linux® VFS.
  • Motivation:
    • Much configuration/convention needs to be accessed by multiple hosts/machines.
    • A significant subset of this information has appeared more recently due to increased use of VM guests and software repositories (mirrors) aboard Linux servers.
      • Especially for files/directories containing:
        • VM information (configurations, disk image files)
        • software repositories/metadata (yum caches, channel dumps)
        • certain types of build trees (i.e. mocks)
        • database files that may be used in more than one machine context
        • mail spools
        • enclave-wide security settings
    • This data is currently distributed in files that can seem scattered, due to a preexisting legacy directory schema adopted from Unix® (/home, /usr, /etc).
    • They could be collected by a more useful schema reflecting utility/purpose/application, however...
    • Parts of current filesystem schema may not work well as this collection place.
      • e.g. /var cannot be made nearly "machine-context-free":
        • /var/run -- holds machine dependent process information,
        • /var/lock -- ...same,
        • /var/spool/mail -- ...is really application-specific, not machine-specific.
      • e.g. /etc doesn't fit, either, as it contains a mixture of "app-machine-context", "machine-context", and "app-context".
        • /etc/qemu -- app-context, although much of /etc is host-specific data,
        • /etc/libvirt/qemu -- VM guest context, not host context,
        • /etc/sysconfig -- however, is machine-context, like much of /etc.
    • Therefore, designation of a new location for such files/directories can be helpful to users.
      • These files can be moved, and then symlinked back into their original places.
      • This designation and use can only be worthwhile if performed without significant disruption to the well-known (and FHS-specified) Linux filesystem tree.
    • Moreover, an analysis of file commonality indicates that creation of a convention of a new colocation affords an opportunity of utility. Filing the data in a manner that encourages collection by purpose (application) also enhances visibility to potential users. This is done by refactoring portions of the hierarchy according to purpose (machine-context-free application data) as opposed to a normal directory hierarchy (which has some machine-context-specific schema that can hide and scatter common data if applied past the symlink).
    • Moreover, such a new location will likely contain files with similar security ACLs, promoting a more fitting default file-creation ACL for that location. This utility is most especially noticed as useful among users of virtual machine images created (or moved to) directories that contain ACLs too restrictive for presentation to a hypervisor (even though the ACLs are generally appropriate).
  • Convention:
    • context -- part of a directory path/tree schema precipitated or influenced by machine (OS or filesystem specific) requirement or convention.
      • a path that reflects some combination of OS/distro/sw/version,
      • aka MCF -- Machine Context-Specific.
    • non-context -- may (perhaps somewhat needlessly) reflect some machine context, but could be structured more usefully if removed from the bounds of the machine-specific context,
      • aka MCF -- Machine Context-Free.
    • axis -- the segment of a machine-context pathname betwen the above that can be multi-machine-mapped
      • the most specific item common to all contexts
      • all elements before the chosen axis are machine-context-free
      • all elements after the chosen axis are machine-context-specific
      • e.g. for /var/cache/yum/x86_64/, [[move this below]]
        • the MCS part is '/var/cache',
        • the MCF part is 'x86_64/13' (due to the difference between how RHEL and Fedora map their yum caches),
        • thus, the axis is '/yum'.
        • Over time, standardization/convention causes fewer items to remain on the CS portion, and thereby become pivotable.
    • fixture -- the MCS part of a path, kept due to the differences in context.
    • pivot - the MCF part of a path.
    • tovip -- a fully-reversed MCF part of a path (please excuse my presumption of neologism for the reverse spelling of pivot).
    • mmmm -- a multi-machine mappable magazine (the thing whose name will be mmm)
      • the actual magazine containing actual data,
      • contents could be non-OS-specific,
      • contents could be non-app-version-specific,
      • could be a volume,
        • could contain a filesystem,
          • filesystem subdirs could be done as:
          • [dirname] -- mappable dirs,
          • [appname] -- common app or package names,
          • [axis]
            • elements before the axis are pivoted/reversed on the mmmm and appended to the axis
            • [/botdir/middir/topdir] is mapped into via symlink from machine context wanting a /topdir/middir/botdir when [botdir] is an [appname] axis
          • [OS_ver] -> [dirname]
          • [SW_ver] -> [dirname]
          • .mmm
            • a file describing specific available mappings
            • each mapping could describe OS- or software-version-specific mappings when non-eliminable
            • format (LON)
    • /mmm - a mapping for a set of axes (colloquially: a remapping)
      • top level pivot dir used in a machine context for mapping
      • could be a direct mount, i.e.:
        • /dev/sdn on /mmm
        • /dev/mapper/mmmm on /mmm
      • could be a symlink into an actual mount point, i.e.:
        • /mmm > /media/volname
          • when /dev/sdn on /media/volname
      • apps could symlink to the top axis, e.g.:
        • [pivot]/[axis]/[fixture] -> [mmm]/[axis]/[tovip]/[fixture]
        • /var/lib/libvirt -> /mmm/var/lib/libvirt
        • /var/lib/libvirt -> /mmm/libvirt/lib/var
        • /etc/libvirt -> /mmm/libvirt/etc
        • /var/cache/yum -> /mmm/yum/cache/var
        • /var/lib/yum -> /mmm/yum/lib/var
        • /var/cache/yum/x86_64/6Workstation/ -> /mmm/yum/cache/var/x86_64/6Workstation
  • Usage:
    • path/dir naming conventions:
      • for package repos:
        • [distro]/[distro-version]
        • /var/cache/yum/x86_64/13
      • for compatibility assemblies:
        • [arch]/[exe-ABI-format]/[exe-load/linkability-format]/[exe-dependent-library-matrix]

Freedom Had A Shitty UX

The snares of the mind were never sprung like traps.  They came slowly, like a growing ivy...with flowers.  Even that metaphor isn't good enough, because ivy can be unwelcome.  The prisoners never complained about these walls.  Hell, they actually paid to have them all set up.  Why not?  Can a well-stocked cabin on a tropical island be called a prison?  It was even more insidious than that really, more like a tropical island with a hospital, teleport, and all your friends there.  And your bank, oozing with cash.  Even the office. Very carefully, tastefully decorated.

But their thoughts and memories are all trapped.  Instantly accessible, indexed, available anywhere, backed up, annotated, mobile, shareable, easily changeable, just a gesture away, really.  Inside. In protected, partitioned memory spaces.

And, more or less importantly, all communication with others.  Speech.  Writing.  Documentation.  Contracts.  EULAs.  Whispers.  Smileys.  Even wishes in absentia, like DRM.  While protecting the Barreds in the righful use of their work, that protection is only given and invoked by the silent assent of...the snare.

Inside the snares, between the snares, protected by the snares, directed by the snares, among the snares.

The snares never say "no", although they could. Theoretically.

A few crackpots in the tundra complained.  Dirty, smelly, miserable lot living in camps, mostly.  Staring at flames all day.  They said there was once a time before, when people could choose for themselves.  And now it was too late, they couldn't free themselves.  That at any time, someone else can wave a multifingered gesture, and take it all away.  Your work, savings, the things that entertain you.  Your collected and carefully distilled thoughts.  Even your own precious memories.  And anything that counts as a meaningful legacy.  As much as they liked, when they liked.  And they could tell you when to come and go.  They simply choose not to...yet.  But "you were warned!...even the logo had a bite taken out of it." was one of their favorite chides.  And something about how nobody will ever notice the gentle nudges....not even a breath of wind. 

"What's more important," they said: "A prosthetic for the mind that could say 'no' to you...but almost never does (well, only by mistake, of course)?  Or the 'trap' of having all the thoughts still in the meat of the mind, or at best a lot less mobile among those...but unassailably, reliably, and deterministically yours?"

And how the stakes were much higher than people realized, given the multiplying power of how the mind prostheses could be used to synthesize the thoughts of many...even all mankind.  Incomparable, unheard of wonders are coming, and someone wants to put a meter on them.  So just be open.

A few people bothered to argue with them that "things were balanced" and such, so the iNtrapment is iMpossible.  If what the Openeers and the Effers said was true, even their message would never get out to the supposedly iMprisoned, right?  So their argument must be self-defeating.  Thank heaven the iNslaved and the iToos could argue from the comfort of their iSlands, in the spare time left by their time-saving snares, sipping Pinas ordered over the iNternet.  But who cares...after a time, even the self-determinate hermits couldn't eat unless their own pitiful (but free, don't forget!) prostheses bowed down while thinking with the snares...in their own language.  Or so they iMplied.

The people who brought the snares to the happy masses were lauded.  Honored.  For improving lives, saving time, and bringing safety.  And quarterly, low-cost upgrade announcements for the stockholders, too.  Called heroes, dreamers, entrepreneurs, visionaries.  And, in a simultaneously multifaceted gem of historical consistency, and historical blasphemy....American. iRonically.

One of the most prominent iNnovators of these iNfernal iNventions even once said that "saving 30 seconds from each device we sell adds up to the equivalent of a human life!"  Well, now that guy is dead, and the lives he theoretically saved are all there, safe and happy.  Well fed, even, from the productivity gains.  Really happy, actually.  Because happiness is a choice, and it cannot be faked, regardless of where the mind is.  Or where it's allowed to go.  Good thing, since those happy minds are iRrevocably trapped, for iNfinity.  There will be no migration.

The walled garden was called beautiful, and everybody wanted one.  But nobody wanted Freedom, even when it cost nothing.  If they did, Freedom would have an 'i' put in it, and sold.  But they don't.  So it isn't. 

Freedom had a shitty UX.