mii/wasmer
1
0
Fork 0
mirror of https://github.com/mii443/wasmer.git synced 2025-12-06 04:38:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
14,898 Commits 1 Branch 0 Tags
0b44bf1aa7dd6cb15150e40f48789ed88b572cfe
Commit Graph

241 Commits

Author SHA1 Message Date
Ivan Enderlin
d0a95ac81b fix(vfs) Fix 4f5908392. 2021-08-30 15:56:54 +02:00
Ivan Enderlin
e0a18a38cf fix(vfs) Better handle path canonicalization on Windows. 
`C:\` is considered as a root, but we expect it to fail because we
don't accept such path as valid for the moment.
 2021-08-30 15:37:21 +02:00
Ivan Enderlin
dac47a8dcc fix(vfs) Fix inode_of on Windows. 
Calling `as_os_str` on `Component::RootDir` returns `\\`, while we
expect `/` on Unix. A better way is to pattern match against
`Component::RootDir` instead of comparing the `OsStr` in this case.
 2021-08-30 15:03:13 +02:00
Ivan Enderlin
4f59083925 fix(vfs) Remove a Clippy error. 2021-08-30 12:12:08 +02:00
Ivan Enderlin
ac95425a97 test(vfs) Turn on host-fs _and_ mem-fs on by default. 2021-08-30 11:31:04 +02:00
Ivan Enderlin
0613f87c95 fix(vfs) Metadata returns 0 for accessed & co times on error. 2021-08-30 10:40:37 +02:00
Ivan Enderlin
2df47f6272 fix(vfs) Correct support of Windows for FileDescriptor. 2021-08-27 15:46:15 +02:00
Ivan Enderlin
bea626bd4f fix(vfs) Add the no-time feature if std::time is not available. 2021-08-27 14:56:30 +02:00
Ivan Enderlin
31f5c1a9a9 fix(vfs) Fix a type error on Windows. 2021-08-27 14:35:52 +02:00
Ivan Enderlin
047256f937 feat(vfs) Continue the rewrite of the in-memory filesystem. 2021-08-27 14:34:44 +02:00
Ivan Enderlin
9e6a427449 feat(vfs) Continue the rewrite of the in-memory filesystem. 2021-08-27 14:34:19 +02:00
Ivan Enderlin
e617937089 feat(vfs) Rewrite the in-memory filesystem. 
The initial in-memory filesystem, aka `mem_fs`, suffered from several
design issues: dead-locks, data duplications, non-optimal look
up/insertions/deletions algorithms, many unimplemented API etc. This
commit is the first step for the rewrite of `mem_fs` to adopt a new
design.

The new design is closer to `ext4` philosophy. A filesystem is a slab
(think an optimised memory allocations vector,
https://en.wikipedia.org/wiki/Slab_allocation) where indexes represent
inodes, and values are of kind `Node`.

A `Node` is now no longer directly recursive (it's indirectly recursive):

```rust
enum Node {
    File { … },
    Directory { children: Vec<Inode>, … },
}
```

To represent children, a node directory doesn't contain `Node` but
`Inode`. To traverse directories, we must jump from the filesystem'
slab, to the node, to the filesystem' slab etc. This design allows
very quick insertions and deletions (it's a O(1) operation, just
insert in the slab and update the parent's children). It also allows
to traverse the filesystem much quicker because we don't need to
traverse all the nodes, only the children of the currently traversed
directory, which reduces the iterations dramatically.

Apart from this new design, the code is adapted largely, with no
`unwrap`, more error handlings, no more `unimplemented!()`s or
`todo!()`s etc.

Some implementations were also wrong, like:

* Reading from `Stdin` never moved an internal cursor, so the user was
  reading the same data again and again.

* It was possible to read from or to write into a file that doesn't
  have the appropriate permissions (e.g. writing into a file with
  read-only permissions),

* The `append` and `truncate` flags on files weren't supported,

* No path canonicalization,

* and so on.
 2021-08-27 14:20:04 +02:00
Ivan Enderlin
743e302401 test(vfs) Add test cases for mem_fs::FileSystem::remove_dir. 2021-08-19 12:48:23 +02:00
Ivan Enderlin
56d2cf05e0 fix(vfs) Do not unwrap in remove_dir. 
In 2 cases:

1. When the path has a parent that doesn't exist,
2. When the directory doesn't exist.
 2021-08-17 16:01:25 +02:00
Ivan Enderlin
3017a37532 test(vfs) Add tests for FileSystem::craete_dir. 2021-08-17 15:57:12 +02:00
Ivan Enderlin
20d20ed857 fix(vfs) Do not unwrap in create_dir when the path has a parent not registered. 2021-08-17 15:57:02 +02:00
Ivan Enderlin
ff9c7f17c9 fix(vfs) Change FileDescriptor to be unit. 2021-08-17 14:51:25 +02:00
Ivan Enderlin
9e6c07dd9a Fix merge conflict. 
When rebasing, Git losts some of the commit. Here is a bunch of them
in a single commit… To learn more, see
f211130a57.
 2021-08-17 14:44:09 +02:00
Ivan Enderlin
ab83efebe3 Fix merge conflict. 2021-08-17 14:15:51 +02:00
Ivan Enderlin
5583631a0a Merge branch 'js-api-wasi' into feat-vfs-unified-fd 2021-08-17 14:15:24 +02:00
Ivan Enderlin
f211130a57 feat(vfs) Unified API. 
Unfortunately, this patch contains multiple features.

First off, it renames the types from `host_fs` and `mem_fs` so that
they use the same names. Apart from the module name, the types owned
by the module are the same, which simplifies the usage greatly.

Second, the code has been cleaned up a little bit. The more important
clean up is probably the renaming of `MemKind` to `Node`. It's OK to
do that here, because the code is not released yet.

Third, a new `FileDescriptor` type is introduced. It represents a
“generic” file descriptor, that a `VirtualFile` can hold. It contains
a value with the size of the larger size a file descriptor on any
platform, namely on Windows where a file descriptor is the size of a
pointer (compared to Unix where it's a `i32`). Then a bunch of
`TryInto` implementations are written to convert it to `RawFd` (on
Unix) or `RawHandle` (on Windows). I'm not confident it's the best
design for the moment, but it's better than before where Windows file
descriptors were impossible to represent. Note that we cannot make
`FileDescriptor` generic over the system for 2 reasons

1. We can activate multiple file systems at the same time, so the
   representation of a file descriptor can change between file system,

2. If we make `FileDescriptor` generic, then the size of `dyn
   VirtualFile` is unknown at compile-time, which makes impossible to
   use in `wasmer-wasi`, hence this design decision of taking the
   larger possible size for a file descriptor so that it fits all
   scenarios.
 2021-08-17 14:09:41 +02:00
Ivan Enderlin
01724637ae feat(vfs) host_fs and mem_fs are not mutually exclusive. 2021-08-17 09:25:33 +02:00
Ivan Enderlin
0a1ad75e55 Merge branch 'js-api-wasi' into fix-vfs-make-features-exclusive 2021-08-17 09:24:26 +02:00
Syrus Akbary
982735845b Merge pull request #2522 from Hywan/fix-vfs-remove-vfs 
fix(vfs) Remove the commented `vfs` virtual FS
 2021-08-16 10:22:23 -05:00
Syrus Akbary
29af515a54 Merge pull request #2525 from Hywan/feat-vfs-host-tryfrom-metadata 
feat: Replace `fs_metadata_to_metadata` by a `TryInto` implementation
 2021-08-16 10:19:55 -05:00
Ivan Enderlin
fde80a8e5f chore(vfs) Use fs rather than std::fs. 2021-08-16 16:23:11 +02:00
Ivan Enderlin
6ff50be74d feat: Replace fs_metadata_to_metadata by a TryInto implementation. 
This patch replaces the `fs_metadata_to_metadata` function by a
`TryInto` implementation.
 2021-08-16 16:19:31 +02:00
Ivan Enderlin
52be548680 feat(vfs) Make host_fs and mem_fs mutually exclusives. 
This patch generates a compiler error if `host_fs` and `mem_fs` are
both enabled at compile-time. It also generates a compiler error if
none of them have been enabled.
 2021-08-16 15:48:48 +02:00
Ivan Enderlin
f639dd91f2 fix(vfs) Make host_fs the default file system. 2021-08-16 15:48:26 +02:00
Ivan Enderlin
ea26266d52 fix(vfs) Remove the commented vfs virtual FS. 
This filesystem is not used and probably broken. We can safely remove it.
 2021-08-16 15:43:01 +02:00
Syrus Akbary
67e24c3fc8 Update lib/vfs/src/host_fs.rs 
Co-authored-by: Ivan Enderlin <ivan@mnt.io>
 2021-08-13 15:18:42 -05:00
Syrus Akbary
55ff1424d2 Fix compilation in windows 2021-07-26 20:51:10 -07:00
Syrus Akbary
75340e1080 Moved rename_file to the filesystem. Fix fs readdir 2021-07-26 20:27:22 -07:00
Syrus Akbary
185570000f Fix compilation 2021-07-24 12:56:59 -07:00
Syrus Akbary
c810940ee3 Remove WASI rename_file default implementation 2021-07-24 12:49:38 -07:00
Syrus Akbary
f15f9de221 Fixed most TODOs 2021-07-24 12:41:12 -07:00
Syrus Akbary
b2b576fbd4 Removed all warnings 2021-07-24 12:16:58 -07:00
Syrus Akbary
e79b8a075a Make WASI serialization optional 2021-07-24 12:09:04 -07:00
Syrus Akbary
fb616cf048 Improved default fs backing 2021-07-24 11:53:05 -07:00
Syrus Akbary
576312e6c8 Get serializer back into working 2021-07-23 23:50:19 -07:00
Syrus Akbary
4200f2e79a Base support for WASI in wasmer-js 
commit b6245ddd19077015a974e9ffd28f164fedb87801
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 23:07:45 2021 -0700

    Remove unused code

commit 6bbf28c5c6d8c4603606e21af3190c52f41cbeb8
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 22:50:50 2021 -0700

    Remove set_last_accessed, set_last_modified, set_created_time from the VirtualFs

commit 999d6941c93f0ea8c8c803c7b1843ec90bd178b5
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 22:42:23 2021 -0700

    Remove webc files

commit 197b72a8978a2364145e20ebd6d44b5bd23082db
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 22:38:29 2021 -0700

    Revert "Wip updates for python webc"

    This reverts commit 2d3e013e64424c0a50f32f44ad0401680d31bf9d.

    # Conflicts:
    #	Cargo.lock
    #	lib/cli/Cargo.toml
    #	lib/cli/src/commands/run.rs
    #	lib/cli/src/commands/run/wasi.rs
    #	lib/vfs/Cargo.toml
    #	lib/wasi/Cargo.toml
    #	lib/wasi/src/state/builder.rs
    #	lib/wasi/src/state/mod.rs

commit 83b7e3b1a1708ed34694e9a44902bd6d0617f0a4
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 22:34:36 2021 -0700

    revert Remove webc

commit 2de7f7f0ce518c74912509ef1fa4b7b1f00c4d11
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 22:31:05 2021 -0700

    Improved testing

commit 97075c5fa69fd2ef9192a4a9737400359e0ee37f
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 22:06:08 2021 -0700

    Improved bindings

commit 9388790c2d31654701dd2925d30bfb35b158727c
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 18:15:43 2021 -0700

    Renamed virtual-fs to vfs

commit 41d91328cd83d10c0fa7242fcfd731335bcd741e
Merge: c0634645a 51fdf66d5
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 18:00:49 2021 -0700

    Merge branch 'master' into js-api-wasi

commit c0634645aebdf25fb545c63368114b531934970d
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 15:06:08 2021 -0700

    Fixed serde dependency

commit 02def893ae24f2da279ed6d3b39b4e50c18ec708
Merge: f13e4c86d d3930be60
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 23 15:04:35 2021 -0700

    Merge branch 'js-api' into js-api-wasi

    # Conflicts:
    #	lib/wasi/Cargo.toml

commit f13e4c86d48a0a527fa72270f29c9f8b97a6f820
Merge: f4ac010e1 fe04ea894
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 16 00:01:19 2021 -0700

    Merge branch 'js-api' into js-api-wasi

commit f4ac010e1ff33cd046474a500bb5b9e4604085f6
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jul 15 22:50:09 2021 -0700

    Use latest resolver for namespace

commit 505896a47c7d658cb4e39ab5256c9e960a2fedc6
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jul 15 22:49:48 2021 -0700

    Updated WASI implementation

commit 30470cb81248a388ea5cf27f1838aeb264dc8956
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jul 15 15:51:56 2021 -0700

    Improved logging

commit caf0c8289ad8ce2471558baab17adee4abc011f5
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jul 15 13:55:43 2021 -0700

    Improved wasmer-js debugging

commit 4e2278dafe8999c262a0cea5afd23f8ae895abcb
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jul 15 08:29:35 2021 -0700

    Added wasi tests

commit 4f5c1fdbdae1f953498016c32d158038b6a6dc97
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 22:22:41 2021 -0700

    Fixed build

commit 692111d098b73229dbe722645b3d86ff6f5e4a5e
Merge: 35dade1b1 7195a9c3f
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 22:18:01 2021 -0700

    Merge branch 'js-api' into js-api-wasi

    # Conflicts:
    #	lib/js-api/src/trap.rs

commit 35dade1b16f01ce67a23ba4fbfa25224dda1403e
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 22:12:38 2021 -0700

    Fixed wasi wast

commit d3a6b42f8ebd790024c7631a87a295bd6957bb80
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 22:12:22 2021 -0700

    Added extra instruction for wasmer-js

commit 95b5c1ad005b17a3bacca45f906d717780f4f95a
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 22:12:12 2021 -0700

    Improved API to be resilient in and out js

commit e4074de27bd4adcd9ee8e9e4973c2cfd5208d523
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 18:54:14 2021 -0700

    Improved traps

commit e512e06cc8c476a74eb9a8e381b0d4019cdc4365
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 18:31:30 2021 -0700

    Improved WasiFs

commit cc7bda3ae33a01336e79e0a6ca48e2a72eb0ad98
Merge: 0721bbfe0 8047e3eb6
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Wed Jul 14 15:15:05 2021 -0700

    Merge branch 'js-api' into js-api-wasi

    # Conflicts:
    #	lib/js-api/Cargo.toml
    #	lib/js-api/src/cell.rs
    #	lib/js-api/src/env.rs
    #	lib/js-api/src/export.rs
    #	lib/js-api/src/exports.rs
    #	lib/js-api/src/externals/function.rs
    #	lib/js-api/src/externals/memory.rs
    #	lib/js-api/src/externals/table.rs
    #	lib/js-api/src/instance.rs
    #	lib/js-api/src/lib.rs
    #	lib/js-api/src/module.rs

commit 0721bbfe04d8d3b54ddcfa66a60a3cb501631ac6
Merge: 8354e03f1 3604debec
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 2 16:15:49 2021 -0700

    Merge branch 'master' into js-api-wasi

commit 8354e03f19b810f8fdaf0cadc3eb89bad30220ab
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 2 10:25:27 2021 -0700

    Improved sync

commit bfa6db7e7201b46fd992361699141df501e6d13c
Merge: 30aa8d4bb 7553efba3
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jul 2 10:07:23 2021 -0700

    Merge branch 'features-cell' into js-api-wasi

    # Conflicts:
    #	lib/api/src/ptr.rs

commit 30aa8d4bb0115b3bb8275212d2f33327d32546e6
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jul 1 17:36:47 2021 -0700

    Improved VirtualFS

commit a72dc93d345ca2c4967a75a0e3ef5243ebe86f33
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jul 1 17:36:37 2021 -0700

    Improved wasmer-js API

commit 63c13f371f19103adb74a6f394c4c09d8fcc4d52
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 17:32:33 2021 -0700

    Updated memory_fs

commit a9e7206b7437984ac367351ea75bb14c8cafaf4d
Merge: 6e7285af0 be5af68ef
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 17:14:29 2021 -0700

    Merge branch 'feature/webc' into js-api-wasi

    # Conflicts:
    #	Cargo.lock
    #	lib/cli/Cargo.toml
    #	lib/virtual-fs/Cargo.toml
    #	lib/wasi/Cargo.toml
    #	lib/wasi/src/state/mod.rs

commit 6e7285af02367a6431b302c17b708f8e0854be82
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 17:07:39 2021 -0700

    Updated virtual fs deps

commit 5eab9ac6f629ddb82f5e97c5fce37998fadf262d
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 17:07:29 2021 -0700

    Make wasi not serializable

commit 1d242d4564e898d773601b87d135a8f5dc4a0f57
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 17:05:30 2021 -0700

    Set WASI to use Strings instead of str

commit 1d7cd64232e4bdd57b64d884d4d9b38dd68e1cdb
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 17:03:23 2021 -0700

    Improve Virtual FS

commit 85f4bbe5ebc0e87138e74a43665616e96772ac5d
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 16:57:14 2021 -0700

    wasmer-js: Added functions to the mix

commit 27effe1eda37a1d810a80a489ff5c737e986f200
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 16:01:12 2021 -0700

    wasmer-js function environments working

commit be5af68efe44c5084dd140f9cc6b425f61a84f8a
Author: Mark McCaskey <mark@wasmer.io>
Date:   Fri Jun 25 15:01:18 2021 -0700

    Move webc vfs impl into wasmer src tree

commit 765c89bb3954699b86a2b820fd40857811e6c466
Author: Mark McCaskey <mark@wasmer.io>
Date:   Fri Jun 25 14:38:19 2021 -0700

    Get python.webc working

commit bedc25cbf00a91cca193bb21834117dc3344febe
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Fri Jun 25 09:17:26 2021 -0700

    wasmer-js improve get_host_environment

commit ee3a36d53b06be72dc00877332c4aaf0b2f57aa4
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jun 24 19:37:34 2021 -0700

    Added wasm32 API

commit c136c2ac6829e4ac3d916dc5687a7c999dd6c412
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jun 24 18:56:19 2021 -0700

    Improved wasmer_js API

commit f819cbbb629442f6f020985ca34b2b57bc90deca
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jun 24 18:32:29 2021 -0700

    Improved wasmer_js imports

commit 1d4ed08d90808922ee5e74e92f162509ba8c2efb
Merge: 5451d7113 2d3e013e6
Author: Syrus Akbary <me@syrusakbary.com>
Date:   Thu Jun 24 17:55:25 2021 -0700

    Merge branch 'feature/webc' into js-api-wasi

    # Conflicts:
    #	lib/wasi/src/syscalls/mod.rs

commit 2d3e013e64424c0a50f32f44ad0401680d31bf9d
Author: Mark McCaskey <mark@wasmer.io>
Date:   Thu Jun 24 16:54:43 2021 -0700

    Wip updates for python webc

commit 3a5a687964780d48e8598247009721badd50d496
Author: Mark McCaskey <mark@wasmer.io>
Date:   Wed Jun 23 10:36:06 2021 -0700

    Add patch to get vfs
 2021-07-23 23:10:17 -07:00