'How to add layer on top of file system? Like cache in OneDrive or Google Drive

I'm looking for a way (using C# .Net or C++ and WinApi or anything) to add a layer between filesystem and client application to add custom behaviour.

Like OneDrive and Google drive application on Windows. You can browse files normally but if given file is missing from a drive it will be downloaded. I need to do something similar. And I need this solution to be transparent to other software (explorer and any other software that accesses files on drive).

I really even know what to call it. So if you have a name for that type of functionality please let me know.

c++.netwindowswinapifilesystems


Solution 1:[1]

Here are some approaches to building a virtual file system on Windows:

  1. Using Cloud Files API. OneDrive is built using this API. It is designed for slow-speed remote (cloud) storage, such as Document Management System. Here are some features that it provides:

    • On-demand loading. You can load the file content into the local drive only when an application is accessing it the first time. The folder listing can be done during the first access to the folder.
    • Offline files support. You can pin/unpin files to keep on the local drive when the server is unavailable.
    • Download/upload progress reporting.
    • Integration with Windows File Manager. Each item shows offline status, pinned/unpinned status, in-sync status, and download progress.
    • Messages and error notifications to the user, as well as file system status reporting.

    The driver that this API is using is installed with Windows by default and does not require installation. Here are some tools to build an OneDrive-like application:

  2. Using Projected File System (ProjFS). It allows you to represent (project) some hierarchical technical data as a file system. It is designed for high-speed back-end storage, such as for example registry. Here is its major feature:

    • Hides the fact that the data is remote. The user "thinks" that the files are in the local file system and does not have any status information. The user does not know if the file content is available/unavailable or online/offline.

    ProjFS is not installed by default and requires Windows component installation (thank you IInspectable for this info). Here are some tools to build with it:

  3. Using Shell Namespace Extensions. Typically used to create a UI extension for Windows File Manager. For example, Windows Recycle Bin and Printers are namespace extensions. They are not available via standard Windows File API and applications can not read/write to it unless it redirects to a real file system behind it. Here some features that it provides:

    • You can customize the Windows File Manager toolbar, shortcut menus, folder view, tree view, and status bar.
    • You can extend the Windows File Manager hierarchy, adding new custom nodes that look and behave like files and folders (cut/copy/paste, drag-and-drop, etc), without building a functional file system behind it.

    Shell Namespace Extensions are part of Windows and does not require additional components installation. In many cases, you will use Shell Extension to add shortcut menus and items on a toolbar in Windows File Manager while building the actual file system using Cloud Files API or ProjFS. Links to start working with Shell Namespace Extensions:

Solution 2:[2]

What you are looking for is commonly called a "Virtual File System". To applications it looks like a file system, but the implementation is at your discretion, allowing you to materialize objects in any way you see fit.

On Windows you have several options to implement a virtual file system. The most common and widely supported infrastructure is the Shell Namespace. It allows you to register extensions, that appear as items (e.g. files or folders) in the file system. It's not entirely trivial to write a Shell Namespace Extensions, but it's not rocket science either.

A more recent addition to Windows is the Projected File System, that provides everything you need to build a virtual file system in user code. It ships as an optional component in Windows, and needs to be explicitly installed first. For an implementation that uses the ProjFS, see the VFS for Git github repository.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1
Solution 2 IInspectable

Related Questions