1
0
mirror of https://github.com/ipfs/kubo.git synced 2025-08-24 19:27:14 +08:00
Files
kubo/docs/plugins.md
Hector Sanjuan 4255cc3889 feat: telemetry plugin (#10866)
* Initial pass at Telemetry plugin

Currently, IP Shipyard, with the help of Probelab, monitor and extract
Amino/IPFS public network metrics with the use of DHT crawlers and
bootstrappers (via peerlog plugin). For example, we log all peer IDs seen and
their AgentVersion/Addresses obtained from the `identify` protocol, which
provides insights into protocol usage, total number of peers etc.

We would like to increase the ability to obtain more insights from the network
by collecting some more information in the future, but also to give users more
control over this collection (i.e. opt-out). The information collected will
not allow unique identification of anyone and is only used for aggregation.

Now, this PR explores a way of moving in this direction:

* A new "telemetry" fx plugin is in charge of dealing with telemetry
* The FX plugin allows to plug and make decisions / take actions during the setup phase:
  * We can inspect whether we are using Private Networks before the libp2p.Host has been initialized.
  * We can send telemetry after the libp2p Host is initialized.
  * Everything is self-contained. Custom builds can remove the plugin altogether without needing to surgically edit the code.

As for behaviour:

* The user can opt-in/out via EnvVar, file in the repo path or plugin configuration.
* Users on private networks or with custom bootstrappers are detected, offered a wall of text explaining why we need telemetry and invited to opt-in. Opt-out happens otherwise on a timeout (with no input). Their preferences are stored.
* Users on standard settings are opted-in by default. This is the status quo in Kubo already, except they don't get a chance to opt out.

The telemetry libp2p protocol is yet to be defined, but expect something similar to identify, with a protobuf being pushed to bootstrappers or to a specific telemetry node that we define. In the case of pnets, this will be done with a temporary peer.

* checkpoint

* telemetry plugin: second pass

* On first run it generates a UUID and shows a message to the user.
* UUID is persistend to "telemetry_uuid"
* Sends telemetry 1 minute after boot and every 24h
* LogEvent is the thing containing all the telemetry that is sent
* Opt-out possible via env-var or plugin configuration

* Telemetry: add changelog and environment variable documentation

* docs: improved daemon message

making it more obvious nothing was sent yet
and that user had 15m to out-out

plus some debug logs that confirm opt-out

* refactor: rename IPFS_TELEMETRY_MODE to IPFS_TELEMETRY

* fix: add User-Agent header to telemetry requests

---------

Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-18 20:46:05 +02:00

204 lines
6.7 KiB
Markdown

# Plugins
Since 0.4.11 Kubo has an experimental plugin system that allows augmenting
the daemons functionality without recompiling.
When an IPFS node is started, it will load plugins from the `$IPFS_PATH/plugins`
directory (by default `~/.ipfs/plugins`).
**Table of Contents**
- [Plugin Types](#plugin-types)
- [IPLD](#ipld)
- [Datastore](#datastore)
- [Available Plugins](#available-plugins)
- [Installing Plugins](#installing-plugins)
- [External Plugin](#external-plugin)
- [In-tree](#in-tree)
- [Out-of-tree](#out-of-tree)
- [Preloaded Plugins](#preloaded-plugins)
- [Creating A Plugin](#creating-a-plugin)
## Plugin Types
Plugins can implement one or more plugin types, defined in the
[plugin](https://godoc.org/github.com/ipfs/kubo/plugin) package.
### IPLD
IPLD plugins add support for additional formats to `ipfs dag` and other IPLD
related commands.
### Datastore
Datastore plugins add support for additional datastore backends.
### Tracer
(experimental)
Tracer plugins allow injecting an opentracing backend into Kubo.
### Daemon
Daemon plugins are started when the Kubo daemon is started and are given an
instance of the CoreAPI. This should make it possible to build an ipfs-based
application without IPC and without forking Kubo.
Note: We eventually plan to make Kubo usable as a library. However, this
plugin type is likely the best interim solution.
### fx (experimental)
Fx plugins let you customize the [fx](https://pkg.go.dev/go.uber.org/fx) dependency graph and configuration,
by customizing the`fx.Option`s that are passed to `fx` when the Kubo node is initialized.
For example, you can override an interface such as [exchange.Interface](https://github.com/ipfs/go-ipfs-exchange-interface)
or [pin.Pinner](https://github.com/ipfs/go-ipfs-pinner) with a custom implementation by appending an option like
`fx.Decorate(func() exchange.Interface { return customExchange })`.
Fx supports some advanced customization. Simple interface replacements like above are unlikely to break in the future,
but the more invasive your changes, the more likely they are to break between releases. Kubo cannot guarantee backwards
compatibility for `fx` customizations.
Fx options are applied across every execution of the `ipfs` binary, including:
- Repo initialization
- Daemon
- Applying migrations
- etc.
So if you plug in a blockservice that disallows non-allowlisted CIDs, then this may break migrations
that fetch migration code over the IPFS network.
### Internal
(never stable)
Internal plugins are like daemon plugins _except_ that they can access, replace,
and modify all internal state. Use this plugin type to extend Kubo in
arbitrary ways. However, be aware that your plugin will likely break every time
Kubo updated.
## Configuration
Plugins can be configured in the `Plugins` section of the config file. Here,
plugins can be:
1. Passed an arbitrary config object via the `Config` field.
2. Disabled via the `Disabled` field.
Example:
```js
{
// ...
"Plugins": {
"Plugins": {
// plugin named "plugin-foo"
"plugin-foo": {
"Config": { /* arbitrary json */ }
},
// plugin named "plugin-bar"
"plugin-bar": {
"Disabled": true // "plugin-bar" will not be loaded
}
}
}
}
```
## Available Plugins
| Name | Type | Preloaded | Description |
|---------------------------------------------------------------------------------|-----------|-----------|------------------------------------------------|
| [git](https://github.com/ipfs/kubo/tree/master/plugin/plugins/git) | IPLD | x | An IPLD format for git objects. |
| [badgerds](https://github.com/ipfs/kubo/tree/master/plugin/plugins/badgerds) | Datastore | x | A high performance but experimental datastore. |
| [flatfs](https://github.com/ipfs/kubo/tree/master/plugin/plugins/flatfs) | Datastore | x | A stable filesystem-based datastore. |
| [levelds](https://github.com/ipfs/kubo/tree/master/plugin/plugins/levelds) | Datastore | x | A stable, flexible datastore backend. |
| [jaeger](https://github.com/ipfs/go-jaeger-plugin) | Tracing | | An opentracing backend. |
| [telemetry](https://github.com/ipfs/kubo/tree/master/plugin/plugins/telemetry) | Telemetry | x | Collects anonymized usage data for Kubo development. |
* **Preloaded** plugins are built into the Kubo binary and do not need to be
installed separately. At the moment, all in-tree plugins are preloaded.
## Installing Plugins
Kubo supports two types of plugins: External and Preloaded.
* External plugins must be installed in `$IPFS_PATH/plugins/` (usually
`~/.ipfs/plugins/`).
* Preloaded plugins are built-into the Kubo when it's compiled.
### External Plugin
The advantage of an external plugin is that it can be built, packaged, and
installed independently of Kubo. Unfortunately, this method is only supported
on Linux and MacOS at the moment. Users of other operating systems should follow
the instructions for preloaded plugins.
#### In-tree
To build plugins included in
[plugin/plugins](https://github.com/ipfs/kubo/tree/master/plugin/plugins),
run:
```bash
kubo$ make build_plugins
kubo$ ls plugin/plugins/*.so
```
To install, copy desired plugins to `$IPFS_PATH/plugins`. For example:
```bash
kubo$ mkdir -p ~/.ipfs/plugins/
kubo$ cp plugin/plugins/git.so ~/.ipfs/plugins/
kubo$ chmod +x ~/.ipfs/plugins/git.so # ensure plugin is executable
```
Finally, restart daemon if it is running.
#### Out-of-tree
To build out-of-tree plugins, use the plugin's Makefile if provided. Otherwise,
you can manually build the plugin by running:
```bash
myplugin$ go build -buildmode=plugin -o myplugin.so myplugin.go
```
Finally, as with in-tree plugins:
1. Install the plugin in `$IPFS_PATH/plugins`.
2. Mark the plugin as executable (`chmod +x $IPFS_PATH/plugins/myplugin.so`).
3. Restart your IPFS daemon (if running).
### Preloaded Plugins
The advantages of preloaded plugins are:
1. They're bundled with the Kubo binary.
2. They work on all platforms.
To preload a Kubo plugin:
1. Add the plugin to the preload list: `plugin/loader/preload_list`
2. Build ipfs
```bash
kubo$ make build
```
You can also preload an in-tree but disabled-by-default plugin by adding it to
the IPFS_PLUGINS variable. For example, to enable plugins foo, bar, and baz:
```bash
kubo$ make build IPFS_PLUGINS="foo bar baz"
```
## Creating A Plugin
To create your own out-of-tree plugin, use the [example
plugin](https://github.com/ipfs/go-ipfs-example-plugin/) as a starting point.
When you're ready, submit a PR adding it to the list of [available
plugins](#available-plugins).