[#6561] improvement(core): Cache Hadoop Filesystem instance on Gravitino server to improve the performance

[sunxiaojian]

What changes were proposed in this pull request?

Cache Hadoop Filesystem instance on Gravitino server to improve the performance

Why are the changes needed?

Fix: #6561

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

N/A

[sunxiaojian]

@yuqi1129 PTAL, thanks

[yuqi1129]

I will take some time to verify whether this works in the loud filesystem as the related ITs are not triggered by default.

[yuqi1129]

Will this method only be used by the Gravitino server?

[yuqi1129]

What if the config already contains key fs.%s.impl.disable.cache? will we cover it?

[sunxiaojian]

It will overwrite. I think we should prioritize the disableCache parameter.

[sunxiaojian]

I will take some time to verify whether this works in the loud filesystem as the related ITs are not triggered by default.

ok

[yuqi1129]

@xloya @jerryshao I don't see any issues. Can you take a look?

[xloya]

If I understand correctly, the current version of FileSystem Provider will also be used for Hadoop GVFS. I think using Hadoop's own cache here will cause security issues, because the authenticated FileSystem is no longer encapsulated by GVFS, and others can arbitrarily obtain the authenticated FileSystem through FileSystem.get() and do some unauthorized behavior.
If you just want to add a filesystem cache on the server side, I think you can provide a cache class. Otherwise, we need to ensure that the client cannot get the authenticated filesystem at will.

[sunxiaojian]

If I understand correctly, the current version of FileSystem Provider will also be used for Hadoop GVFS. I think using Hadoop's own cache here will cause security issues, because the authenticated FileSystem is no longer encapsulated by GVFS, and others can arbitrarily obtain the authenticated FileSystem through FileSystem.get() and do some unauthorized behavior. If you just want to add a filesystem cache on the server side, I think you can provide a cache class. Otherwise, we need to ensure that the client cannot get the authenticated filesystem at will.

thanks, Let me check it again.

[yuqi1129]

If I understand correctly, the current version of FileSystem Provider will also be used for Hadoop GVFS. I think using Hadoop's own cache here will cause security issues, because the authenticated FileSystem is no longer encapsulated by GVFS, and others can arbitrarily obtain the authenticated FileSystem through FileSystem.get() and do some unauthorized behavior. If you just want to add a filesystem cache on the server side, I think you can provide a cache class. Otherwise, we need to ensure that the client cannot get the authenticated filesystem at will.

On the Gravitino server side, the fs cache is only used by the Gravitino server. Why can't we use the file system cache? @xloya , can you help to clarify it more clearly.

In GVFS client, indeed, there will be a security vulnerability if the fs cache is enabled.

[sunxiaojian]

The GVFS side uses its own maintained cache, and when GVS initializing Filesystem, it will force not to use FS cache

[xloya]

If I understand correctly, the current version of FileSystem Provider will also be used for Hadoop GVFS. I think using Hadoop's own cache here will cause security issues, because the authenticated FileSystem is no longer encapsulated by GVFS, and others can arbitrarily obtain the authenticated FileSystem through FileSystem.get() and do some unauthorized behavior. If you just want to add a filesystem cache on the server side, I think you can provide a cache class. Otherwise, we need to ensure that the client cannot get the authenticated filesystem at will.

On the Gravitino server side, the fs cache is only used by the Gravitino server. Why can't we use the file system cache? @xloya , can you help to clarify it more clearly.

In GVFS client, indeed, there will be a security vulnerability if the fs cache is enabled.

What I mean is that we could enable Hadoop Filesystem cache in the server side, but the current Filesystem Provider is not only designed for the server, but also for the client. I think we need to consider how to enable Hadoop Filesystem cache only on the server, but not on the client.
In addition, the Hadoop GVFS client already has an internal FileSystem cache that is independent of the Hadoop Filesystem cache, so there is no need to use FileSystem.get() to cache the FileSystem to improve performance. Using FileSystem.get() in the Hadoop GVFS client will cause security issues instead.
So from this conclusion, I think there are two ways to solve this problem:

1. Keep the current way of using FileSystem Provider on both the client and the server, and still use Filesystem.newInstance() to always create a new Filesystem. At the same time, add an internal FileSystem cache on the server, just like the implementation of Hadoop GVFS.
2. Consider splitting or modifying the logic of FileSystem Provider to support different logics on the client and server to avoid the security issues of the client's Hadoop FileSystem cache.