Storing huge amount of files is a common problem during Java based development, but a universal solution is not available. Remote file access is often a problem in modern applications, but it needs complex planning, preparation and implementation. Managing the operations of parallel users working on the same data set need can be difficult also. The system has to be able to correctly separate the parallel actions.
A solution available for some part of the problem, but integrating them to a complete system is a complex task. Generally the remote file access needs separate server software, and the support of these protocols are depends on the operating systems. Separating parallel users with transactions in database systems has wide range of exprience, but there in only a few solutions to the transactional file storage. Planning a proper data structure can be difficult for huge amount of files and the developers must be aware of the limitations and specialties of the common file systems, like the maximal number of files in a folder or the useable symbols is file names. File operations can be done with the native Java classes, but generally they are not convenient.
Separating the logical file structure from the physical storage provides a way to implement an efficient file store solution in the background, which is transparent to the developers. This solution allows us to develop advanced functions, like single instance storage or transactional file access. Raising the network layer to the application level can prevent the need of the lengthy preparations and allows the developers to access remote files in the same way as local files.
The aim of the development is to create a universal, operating system independent solution to these problems, which is can be used locally and remotely to store huge amount of files in a convenient way.