Fetch¶
fetch fetches a Resource from an URL and outputs its content. You can therefore use this similarly to curl or httpie or wget. (If you want to see the Media Type, use info detect.)
This is different from get, which shows Thing/s given an _IRI.
(However the --load option of get internally does a fetch, and supports the same schemes.)
Enola supports the URI schemes which are documented below.
These are supported everywhere; including in fetch, --load, and elsewhere.
Schemes¶
Files¶
We can do cat-like equivalent of local files using the file: scheme:
$ echo "hello, world" >/tmp/hi.txt && ./enola fetch file:///tmp/hi.txt
hello, world
We can omit the file: scheme and use absolute or relative paths,
because (in the CLI) the current working directory is implicitly the base URI
used to resolve URI references:
$ ./enola fetch /tmp/hi.txt
hello, world
When running a remotely accessible server, you’ll most probably want to disable the file: scheme,
to block access to local files for security:
$ ./enola fetch --no-file-scheme /tmp/hi.txt
null scheme unknown; try: enola fetch --help
HTTP¶
This will fetch https://www.vorburger.ch/hello.md: (Note how for security reasons we have to explicitly permit it.)
$ ./enola fetch --http-scheme https://www.vorburger.ch/hello.md
# hello, world
Enola locally caches HTTP responses on the filesystem.
IPFS¶
Enola, like e.g. curl, has a “native protocol handler”
to support ipfs: URLs for decentralized content from IPFS:
$ ./enola fetch --ipfs-gateway=https://dweb.link/ipfs/ ipfs://QmXV7pL1CB7A8Tzk7jP2XE9kRyk8HZd145KDptdxzmNLfu
hello, world
The --ipfs-gateway is the URL of an IPFS HTTP Gateway.
Instead of the shown dweb.link,
we do (highly) recommend that you locally install & run an IPFS Node,
such as IPFS Desktop,
or Kubo,
and then use --ipfs-gateway=http://localhost:8080/ipfs/ instead.
For initial testing you can specify one of the public IPFS Gateways (or “rent” one from a provider such as Pinata or Infura or Cloudflare). However, we do NOT recommend using these for anything “real”, because of the following disadvantages:
- Local Nodes make great local caches to improve Enola’s performance
- Public gateways are likely to rate limit your HTTP requests and return 403 errors
- Enola does not verify the CID hash over the fetched content, so you are implicitly trusting the Gateway (and assume no MITM)
- Running an IPFS Node, especially a permanently on and publicly IP reachable one, furthers the global IPFS network
Classpath¶
$ ./enola fetch classpath:/VERSION
fe95192
Data¶
Enola supports (RFC 2397) data: URLs:
$ ./enola fetch "data:application/json;charset=UTF-8,%7B%22key%22%3A+%22value%22%7D"
{"key": "value"}
File Descriptor¶
fd: is a (non-standard) URL scheme in Enola for reading from or writing to file descriptors, for:
The Media Type of this special resource will be application/octet-stream (not application/binary),
unless there is a ?mediaType= parameter.
The Charset will be the default of the JVM,
unless there is (checked first) a ?charset= parameter (e.g. fd:0?charset=UTF-16BE),
or the ?mediaType= parameter includes a charset (e.g. fd:1?mediaType=application/yaml;charset=utf-16be).
Empty¶
empty: is another (non-standard) URL scheme in Enola for “no content” (as an alternative to data:,):
$ ./enola fetch empty:/
Exec¶
TODO We plan to support an exec: scheme, whose content will be the resulting of running the given command,
similar to e.g. 🐪 Camel’s or (vaguely) Web Browsers’
javascript:.
Parameters¶
Enola considers certain generic query parameters in URLs it fetches.
These work with most but not all schemes (e.g. data: does not permit query parameters).
Media Type¶
Adding e.g. ?mediaType=application/json overrides the Media Type
which e.g. a server provided in e.g. a HTTP header,
or that was determined from a file extension.
Charset¶
Adding e.g. ?charset=iso-8859-1 overrides (and takes precedence over)
the Charset from the Media Type (if any) or any HTTP header like mechanisms.
Integrity¶
Adding ?integrity=... verifies resources via a cryptographic digest (“hash”)
using a Multiformats’s Multibase encoded Multihash.
(This is similar e.g. to HTML’s Subresource Integrity (SRI).)
It works for all schemes:
$ ./enola fetch "/tmp/hi.txt?integrity=z8VxiEEn4n7uuGrVQjeoH2KYypytUHttCubqN7rr65xSH3wjLDjHciXuTyTHkoRuJT1Njghj68RQdynADQt9vzLgyEs"
hello, world
or:
$ ./enola fetch --http-scheme "https://www.vorburger.ch/hello.md?integrity=z8VttgvnrXN5ZzqAh8BLwyup7htUmSM9gbKR445teEECTwMRDQTireiWgWauLiZ4Xr5esrqbVFNbAuAM2XyZ4CTxU7N"
# hello, world
In order to find the expected Multibase encoded Multihash, it’s simplest to once use a wrong one, and then replace it with the correct one which is shown by the error message:
$ ./enola fetch --http-scheme "https://www.vorburger.ch/hello.md?integrity=z8VsnXyGnRwJpnrQXB8KcLstvgFYGZ2f5BCm3DVndcNZ8NswtkCqsut69e7yd1FKNtettjgy669GNVt8VSTGxkAiJaB"
Internal Problem occured, add -vvv flags for technical details: IntegrityViolationException: Expected z8VsnXyGnRwJpnrQXB8KcLstvgFYGZ2f5BCm3DVndcNZ8NswtkCqsut69e7yd1FKNtettjgy669GNVt8VSTGxkAiJaB but got z8VttgvnrXN5ZzqAh8BLwyup7htUmSM9gbKR445teEECTwMRDQTireiWgWauLiZ4Xr5esrqbVFNbAuAM2XyZ4CTxU7N
enola info digest is an alternative for obtaining the ?integrity=... value.
Note that while Multihash defines codes for various hash functions,
Enola (currently) intentionally only actually supports sha2-256 & sha2-512.
URL Integrity Spec describes this further.