Tool [Alpha 0.4.0] StarFuse :: .pak utility for Linux/OS X

StarFuse
Mount .pak files as regular directories under Linux and OS X

This tool allows developers to mount StarBound .pak (and .modpak) files as regular directories.

This tool is currently in alpha stage. It should work as a read-only filesystem. The Beta release will include writing to existing Pakfiles, and the first major release will include creating new Pakfiles altogether.

---

Installation
Installation requires that the FUSE libraries be installed on your system. They do require administrator access (usually not a problem) but are otherwise easy to install.

Linux:
(You may need to update these commands to make sense for your particular distribution)

Install pip (python package manager) if you don't already have it:

Code:

$ sudo apt-get install python-pip python-dev build-essential
$ sudo pip install --upgrade pip
$ sudo pip install --upgrade virtualenv

Install FUSE:

Code:

$ sudo apt-get install libfuse-dev

Install StarFuse:

Code:

$ pip install starfuse

Mac OS X
Install pip (python package manager) if you don't already have it:

Code:

$ sudo easy_install pip

Install OSXFUSE via this link (or by visiting https://osxfuse.github.io/ and choosing the appropriate OSXFUSE installation for you).

Alternatively, you may use Homebrew to install OSXFUSE, though I have never really had the best luck with that.

Finally, install StarFuse:

Code:

$ pip install starfuse

---

Using StarFuse
Once installed, it's fairly easy to use StarFuse. Most users will want to use the following:

Code:

$ mkdir ~/pak
$ starfuse /path/to/packed.pak ~/pak

Right now, there is no daemon mode (fairly small change; I just haven't added it yet) so it will stay there until you unmount it:

Code:

$ unmount ~/pak

Make sure no programs have any files open (including any terminals cd'ed into it) prior to unmounting!

For completeness, here is the full help documentation:

Code:

$ starfuse --help
usage: starfuse [-h] [-v] [-w] [--pages PAGES] pakfile mount_dir

Mounts StarBound .pak files as FUSE filesystems

positional arguments:
  pakfile        the .pak file on which to operate
  mount_dir      the directory on which to mount the PAK file

optional arguments:
  -h, --help     show this help message and exit
  -v, --verbose  be noisy
  -w, --write    allow modifications to the .pak file
  --pages PAGES  map this number of pages at a time (default: 256)

---

FAQ

Why would I use this?
Right now, to browse the contents of a .pak or a .modpak file under linux/OS X, you must use Chucklefish's provided asset unpacker utility, which is not only slow but requires that you unpack all of the modpack's contents before you can really do anything with them.

Not only does this process take a while, it also duplicates the files on your system (meaning not only do you have a large .pak file to begin with, you now have a whole directory with exactly the same contents).

Luckily, because Chucklefish wrote their filesystem format correctly (seriously, well done), this leaves .pak files very similar to regular ol' hard disk formats, and thus easy to emulate using FUSE.

This means, without unpacking every file from a .pak file, you can browse the contents of a .pak file as if it were already unpacked into a directory. This includes using image editors, text editors, regular filesystem commands (mv, cp, tar, etc.) on the contents - without ever unpacking them.

Does this work with modpak files?
Short answer: most likely.

Longer answer: I haven't tested it (lack of time). If you have tried it out, let me know if it works/doesn't work in a reply on this thread and I'll update accordingly!

What's the catch?
Right now, you can only read from the pak files. Why? Because writing to them requires some lengthy code that has not yet been written (that's the next step, I promise!)

As well, you have to install FUSE libraries for your system, which requires administrator privileges. Don't worry, they won't interfere with anything and are completely harmless. If you have already used FUSE filesystems before, chances are you don't need to install anything other than Starfuse itself. Neat!

When will I be able to edit Pakfiles?
In Beta

When will I be able to create new Pakfiles?
Short answer: at the first major release.

Slightly longer answer: When modification functionality is added in beta, you could (theoretically) take an existing pakfile, copy it, mount it, and just remove all of the files in it already to create an empty pakfile.

Why release before having modification functionality?
For a few reasons, two of which being to find bugs, as well as to gauge what's more important to the community in terms of functionality. If there is something I can deliver quickly that I didn't think of, I'd rather catch it now than later.

I do have a full time job (and by full time, I mean take-over-my-life job) so the times I can work on Starfuse are pretty limited. Getting people to test it out really helps.

I'm getting a Too many open files error!
Run Starfuse with the --pages flag set to something around 1024.

Code:

$ starfuse --pages=1024 /path/to/pakfile.pak ~/pak

If it still occurs, try setting it even higher - 2048 should be high enough. Anything higher and it's definitely a bug. Most users shouldn't run into this problem unless they're on some crazy bizarre operating system (or, if you're working with really, really big Pakfiles).

I have a problem, where do I post about it?
As of right now, protocol should be to check this thread first to see if it's been brought up. If not, make a reply to this thread about it (mods are welcome to police this however they please).

---

Meta
The source code is available as open source: https://github.com/qix-/starfuse
Documentation is available at CodeDocs: https://codedocs.xyz/Qix-/starfuse/annotated.html

Major props to Blixt for some solid work in this area. The BTreeDB4 implementation as it stands right now (version 0.4.0) is a moderately refactored version of his code. The SBON implementation is, for the most part, completely unmodified (originally by Blixt).

As well, thanks to The Suit for his blessing on this project.

 

[reserved]

 

I'm having issues with it:



For reference, I'm running Python 3.6 on Arch.