4. C API Reference¶

A Futhark program futlib.fut compiled to a C library with the --library command line option produces two files: futlib.c and futlib.h. The API provided in the .h file is documented in the following.

The .h file can be included by a C++ source file to access the functions (extern "C" is added automatically), but the .c file must be compiled with a proper C compiler and the resulting object file linked with the rest of the program.

Using the API requires creating a configuration object, which is then used to obtain a context object, which is then used to perform most other operations, such as calling Futhark functions.

Most functions that can fail return an integer: 0 on success and a non-zero value on error, as documented below. Others return a NULL pointer. Use futhark_context_get_error() to get a (possibly) more precise error message.

FUTHARK_BACKEND_foo¶: A preprocessor macro identifying that the backend foo was used to generate the code; e.g. c, opencl, or cuda. This can be used for conditional compilation of code that only works with specific backends.

4.1. Error codes¶

Most errors are result in a not otherwise specified nonzero return code, but a few classes of errors have distinct error codes.

FUTHARK_SUCCESS¶: Defined as 0. Returned in case of success.

FUTHARK_PROGRAM_ERROR¶: Defined as 2. Returned when the program fails due to out-of-bounds, an invalid size coercion, invalid entry point arguments, or similar misuse.

FUTHARK_OUT_OF_MEMORY¶: Defined as 3. Returned when the program fails to allocate memory. This is (somewhat) reliable only for GPU memory - due to overcommit and other VM tricks, you should not expect running out of main memory to be reported gracefully.

4.2. Configuration¶

Context creation is parameterised by a configuration object. Any changes to the configuration must be made before calling futhark_context_new(). A configuration object must not be freed before any context objects for which it is used. The same configuration may not be used for multiple concurrent contexts. Configuration objects are cheap to create and destroy.

struct futhark_context_config¶: An opaque struct representing a Futhark configuration.

struct futhark_context_config *futhark_context_config_new(void)¶: Produce a new configuration object. You must call futhark_context_config_free() when you are done with it.

void futhark_context_config_free(struct futhark_context_config *cfg)¶: Free the configuration object.

void futhark_context_config_set_debugging(struct futhark_context_config *cfg, int flag)¶: With a nonzero flag, enable various debugging information, with the details specific to the backend. This may involve spewing copious amounts of information to the standard error stream. It is also likely to make the program run much slower.

void futhark_context_config_set_profiling(struct futhark_context_config *cfg, int flag)¶: With a nonzero flag, enable the capture of profiling information. This should not significantly impact program performance. Use futhark_context_report() to retrieve captured information, the details of which are backend-specific.

void futhark_context_config_set_logging(struct futhark_context_config *cfg, int flag)¶: With a nonzero flag, print a running log to standard error of what the program is doing.

int futhark_context_config_set_tuning_param(struct futhark_context_config *cfg, const char *param_name, size_t new_value)¶: Set the value of a tuning parameter. Returns zero on success, and non-zero if the parameter cannot be set. This is usually because a parameter of the given name does not exist. See futhark_get_tuning_param_count() and futhark_get_tuning_param_name() for how to query which parameters are available. Most of the tuning parameters are applied only when the context is created, but some may be changed even after the context is active. At the moment, only parameters of class “threshold” may change after the context has been created. Use futhark_get_tuning_param_class() to determine the class of a tuning parameter.

int futhark_get_tuning_param_count(void)¶: Return the number of available tuning parameters. Useful for knowing how to call futhark_get_tuning_param_name() and futhark_get_tuning_param_class().

const char *futhark_get_tuning_param_name(int i)¶: Return the name of tuning parameter i, counting from zero.

const char *futhark_get_tuning_param_class(int i)¶: Return the class of tuning parameter i, counting from zero.

void futhark_context_config_set_cache_file(struct futhark_context_config *cfg, const char *fname)¶

Ask the Futhark context to use a file with the designated file as a cross-execution cache. This can result in faster initialisation of the program next time it is run. For example, the GPU backends will store JIT-compiled GPU code in this file.

The cache is managed entirely automatically, and if it is invalid or stale, the program performs initialisation from scratch. There is no machine-readable way to get information about whether the cache was hit succesfully, but you can enable logging to see what happens.

The lifespan of fname must exceed the lifespan of the configuration object. Pass NULL to disable caching (this is the default).

4.3. Context¶

struct futhark_context¶: An opaque struct representing a Futhark context.

struct futhark_context *futhark_context_new(struct futhark_context_config *cfg)¶

Create a new context object. You must call futhark_context_free() when you are done with it. It is fine for multiple contexts to co-exist within the same process, but you must not pass values between them. They have the same C type, so this is an easy mistake to make.

After you have created a context object, you must immediately call futhark_context_get_error(), which will return non-NULL if initialisation failed. If initialisation has failed, then you still need to call futhark_context_free() to release resources used for the context object, but you may not use the context object for anything else.

void futhark_context_free(struct futhark_context *ctx)¶: Free the context object. It must not be used again. You must call futhark_context_sync() before calling this function to ensure there are no outstanding asynchronous operations still running. The configuration must be freed separately with futhark_context_config_free().

int futhark_context_sync(struct futhark_context *ctx)¶: Block until all outstanding operations, including copies, have finished executing. Many API functions are asynchronous on their own.

void futhark_context_pause_profiling(struct futhark_context *ctx)¶: Temporarily suspend the collection of profiling information. Has no effect if profiling was not enabled in the configuration.

void futhark_context_unpause_profiling(struct futhark_context *ctx)¶: Resume the collection of profiling information. Has no effect if profiling was not enabled in the configuration.

char *futhark_context_get_error(struct futhark_context *ctx)¶: A human-readable string describing the last error. Returns NULL if no error has occurred. It is the caller’s responsibility to free() the returned string. Any subsequent call to the function returns NULL, until a new error occurs.

void futhark_context_set_logging_file(struct futhark_context *ctx, FILE *f)¶: Set the stream used to print diagnostics, debug prints, and logging messages during runtime. This is stderr by default. Even when this is used to re-route logging messages, fatal errors will still only be printed to stderr.

char *futhark_context_report(struct futhark_context *ctx)¶: Produce a human-readable C string with debug and profiling information collected during program runtime. It is the caller’s responsibility to free the returned string. It is likely to only contain interesting information if futhark_context_config_set_debugging() or futhark_context_config_set_profiling() has been called previously. Returns NULL on failure.

int futhark_context_clear_caches(struct futhark_context *ctx)¶: Release any context-internal caches and buffers that may otherwise use computer resources. This is useful for freeing up those resources when no Futhark entry points are expected to run for some time. Particularly relevant when using a GPU backend, due to the relative scarcity of GPU memory.

4.4. Values¶

Primitive types (i32, bool, etc) are mapped directly to their corresponding C type. The f16 type is mapped to uint16_t, because C does not have a standard half type. This integer contains the bitwise representation of the f16 value in the IEEE 754 binary16 format.

For each distinct array type of primitives (ignoring sizes), an opaque C struct is defined. Arrays of f16 are presented as containing uint16_t elements. For types that do not map cleanly to C, including records, sum types, and arrays of tuples, see Opaque Values.

All array values share a similar API, which is illustrated here for the case of the type []i32. The creation/retrieval functions are all asynchronous, so make sure to call futhark_context_sync() when appropriate. Memory management is entirely manual. All values that are created with a new function, or returned from an entry point, must at some point be freed manually. Values are internally reference counted, so even for entry points that return their input unchanged, you must still free both the input and the output - this will not result in a double free.

struct futhark_i32_1d¶: An opaque struct representing a Futhark value of type []i32.

struct futhark_i32_1d *futhark_new_i32_1d(struct futhark_context *ctx, int32_t *data, int64_t dim0)¶: Asynchronously create a new array based on the given data. The dimensions express the number of elements. The data is copied into the new value. It is the caller’s responsibility to eventually call futhark_free_i32_1d(). Multi-dimensional arrays are assumed to be in row-major form. Returns NULL on failure.

struct futhark_i32_1d *futhark_new_raw_i32_1d(struct futhark_context *ctx, char *data, int64_t offset, int64_t dim0)¶: Create an array based on raw data, as well as an offset into it. This differs little from futhark_i32_1d() when using the c backend, but when using e.g. the opencl backend, the data parameter will be a cl_mem. It is the caller’s responsibility to eventually call futhark_free_i32_1d(). The data pointer must remain valid for the lifetime of the array. Unless you are very careful, this basically means for the lifetime of the context. Returns NULL on failure.

int futhark_free_i32_1d(struct futhark_context *ctx, struct futhark_i32_1d *arr)¶: Free the value. In practice, this merely decrements the reference count by one. The value (or at least this reference) may not be used again after this function returns.

int futhark_values_i32_1d(struct futhark_context *ctx, struct futhark_i32_1d *arr, int32_t *data)¶: Asynchronously copy data from the value into data, which must point to free memory, allocated by the caller, with sufficient space to store the full array. Multi-dimensional arrays are written in row-major form.

const int64_t *futhark_shape_i32_1d(struct futhark_context *ctx, struct futhark_i32_1d *arr)¶: Return a pointer to the shape of the array, with one element per dimension. The lifetime of the shape is the same as arr, and must not be manually freed. Assuming arr is a valid object, this function cannot fail.

4.4.1. Opaque Values¶

Each instance of a complex type in an entry point (records, nested tuples, etc) is represented by an opaque C struct named futhark_opaque_foo. In the general case, foo will be a hash of the internal representation. However, if you insert an explicit type annotation in the entry point (and the type name contains only characters valid in C identifiers), that name will be used. Note that arrays contain brackets, which are not valid in identifiers. Defining a type abbreviation is the best way around this.

The API for opaque values is similar to that of arrays, and the same rules for memory management apply. You cannot construct them from scratch, but must obtain them via entry points (or deserialisation, see futhark_restore_opaque_foo()).

struct futhark_opaque_foo¶: An opaque struct representing a Futhark value of type foo.

int futhark_free_opaque_foo(struct futhark_context *ctx, struct futhark_opaque_foo *obj)¶: Free the value. In practice, this merely decrements the reference count by one. The value (or at least this reference) may not be used again after this function returns.

int futhark_store_opaque_foo(struct futhark_context *ctx, const struct futhark_opaque_foo *obj, void **p, size_t *n)¶

Serialise an opaque value to a byte sequence, which can later be restored with futhark_restore_opaque_foo(). The byte representation is not otherwise specified, and is not stable between compiler versions or programs. It is stable under change of compiler backend, but not change of compiler version, or modification to the source program (although in most cases the format will not change).

The variable pointed to by n will always be set to the number of bytes needed to represent the value. The p parameter is more complex:

If p is NULL, the function will write to *n, but not actually serialise the opaque value.
If *p is NULL, the function will allocate sufficient storage with malloc(), serialise the value, and write the address of the byte representation to *p. The caller gains ownership of this allocation and is responsible for freeing it.
Otherwise, the serialised representation of the value will be stored at *p, which must have room for at least *n bytes. This is done asynchronously.

Returns 0 on success.

struct futhark_opaque_foo *futhark_restore_opaque_foo(struct futhark_context *ctx, const void *p)¶: Asynchronously restore a byte sequence previously written with futhark_store_opaque_foo(). Returns NULL on failure. The byte sequence does not need to have been generated by the same program instance, but it must have been generated by the same Futhark program, and compiled with the same version of the Futhark compiler.

4.4.2. Records¶

A record is an opaque type (see above) that supports additional functions to project individual fields (read their values) and to construct a value given values for the fields. An opaque type is a record if its definition is a record at the Futhark level.

The projection and construction functions are equivalent in functionality to writing entry points by hand, and so serve only to cut down on boilerplate. Important things to be aware of:

The objects constructed though these functions have their own lifetime (like any objects returned from an entry point) and must be manually freed, independently of the records from which they are projected, or the fields they are constructed from.
The objects are however in an aliasing relationship with the fields or original record. This means you must be careful when passing them to entry points that consume their arguments. As always, you don’t have to worry about this if you never write entry points that consume their arguments.

The precise functions generated depend on the fields of the record. The following functions assume a record with Futhark-level type type t = {foo: t1, bar: t2} where t1 and t2 are also opaque types.

int futhark_new_opaque_t(struct futhark_context *ctx, struct futhark_opaque_t **out, const struct futhark_opaque_t2 *bar, const struct futhark_opaque_t1 *foo);¶: Construct a record in *out which has the given values for the bar and foo fields. The parameters are the fields in alphabetic order. Tuple fields are named vX where X is an integer. The resulting record aliases the values provided for bar and foo, but has its own lifetime, and all values must be individually freed when they are no longer needed.

int futhark_project_opaque_t_bar(struct futhark_context *ctx, struct futhark_opaque_t2 **out, const struct futhark_opaque_t *obj);¶: Extract the value of the field bar from the provided record. The resulting value aliases the record, but has its own lifetime, and must eventually be freed.

int futhark_project_opaque_t_foo(struct futhark_context *ctx, struct futhark_opaque_t1 **out, const struct futhark_opaque_t *obj);¶: Extract the value of the field bar from the provided record. The resulting value aliases the record, but has its own lifetime, and must eventually be freed.

4.5. Entry points¶

Entry points are mapped 1:1 to C functions. Return values are handled with out-parameters.

For example, this Futhark entry point:

entry sum = i32.sum

Results in the following C function:

int futhark_entry_sum(struct futhark_context *ctx, int32_t *out0, const struct futhark_i32_1d *in0)¶: Asynchronously call the entry point with the given arguments. Make sure to call futhark_context_sync() before using the value of out0.

Errors are indicated by a nonzero return value. On error, the out-parameters are not touched.

The precise semantics of the return value depends on the backend. For the sequential C backend, errors will always be available when the entry point returns, and futhark_context_sync() will always return zero. When using a GPU backend such as cuda or opencl, the entry point may still be running asynchronous operations when it returns, in which case the entry point may return zero successfully, even though execution has already (or will) fail. These problems will be reported when futhark_context_sync() is called. Therefore, be careful to check the return code of both the entry point itself, and futhark_context_sync().

For the rules on entry points that consume their input, see Consumption and Aliasing. Note that even if a value has been consumed, you must still manually free it. This is the only operation that is permitted on a consumed value.

4.6. GPU¶

The following API functions are available when using the opencl, cuda, or hip backends.

void futhark_context_config_set_device(struct futhark_context_config *cfg, const char *s)¶: Use the first device whose name contains the given string. The special string #k, where k is an integer, can be used to pick the k-th device, numbered from zero. If used in conjunction with futhark_context_config_set_platform(), only the devices from matching platforms are considered.

4.6.1. Exotic¶

The following functions are not interesting to most users.

void futhark_context_config_set_default_group_size(struct futhark_context_config *cfg, int size)¶: Set the default number of work-items in a work-group.

void futhark_context_config_set_default_num_groups(struct futhark_context_config *cfg, int num)¶: Set the default number of work-groups used for kernels.

void futhark_context_config_set_default_tile_size(struct futhark_context_config *cfg, int num)¶: Set the default tile size used when executing kernels that have been block tiled.

const char *futhark_context_config_get_program(struct futhark_context_config *cfg)¶: Retrieve the embedded GPU program. The context configuration keeps ownership, so don’t free the string.

void futhark_context_config_set_program(struct futhark_context_config *cfg, const char *program)¶: Instead of using the embedded GPU program, use the provided string, which is copied by this function.

4.7. OpenCL¶

The following API functions are available only when using the opencl backend.

void futhark_context_config_set_platform(struct futhark_context_config *cfg, const char *s)¶: Use the first OpenCL platform whose name contains the given string. The special string #k, where k is an integer, can be used to pick the k-th platform, numbered from zero.

void futhark_context_config_select_device_interactively(struct futhark_context_config *cfg)¶: Immediately conduct an interactive dialogue on standard output to select the platform and device from a list.

void futhark_context_config_set_command_queue(struct futhark_context_config *cfg, cl_command_queue queue)¶: Use exactly this command queue for the context. If this is set, all other device/platform configuration options are ignored. Once the context is active, the command queue belongs to Futhark and must not be used by anything else. This is useful for implementing custom device selection logic in application code.

cl_command_queue futhark_context_get_command_queue(struct futhark_context *ctx)¶: Retrieve the command queue used by the Futhark context. Be very careful with it - enqueueing your own work is unlikely to go well.

4.7.1. Exotic¶

The following functions are used for debugging generated code or advanced usage.

void futhark_context_config_add_build_option(struct futhark_context_config *cfg, const char *opt)¶: Add a build option to the OpenCL kernel compiler. See the OpenCL specification for clBuildProgram for available options.

cl_program futhark_context_get_program(struct futhark_context_config *cfg)¶: Retrieve the compiled OpenCL program.

void futhark_context_config_load_binary_from(struct futhark_context_config *cfg, const char *path)¶: During futhark_context_new(), read a compiled OpenCL binary from the given file instead of using the embedded program.

4.8. CUDA¶

The following API functions are available when using the cuda backend.

4.8.1. Exotic¶

The following functions are used for debugging generated code or advanced usage.

void futhark_context_config_add_nvrtc_option(struct futhark_context_config *cfg, const char *opt)¶: Add a build option to the NVRTC compiler. See the CUDA documentation for nvrtcCompileProgram for available options.

void futhark_context_dump_ptx_to(struct futhark_context_config *cfg, const char *path)¶: During futhark_context_new(), dump the generated PTX code to the given file.

void futhark_context_config_load_ptx_from(struct futhark_context_config *cfg, const char *path)¶: During futhark_context_new(), read PTX code from the given file instead of using the embedded program.

4.9. Multicore¶

The following API functions are available when using the multicore backend.

void futhark_context_config_set_num_threads(struct futhark_context_config *cfg, int n)¶: The number of threads used to run parallel operations. If set to a value less than 1, then the runtime system will use one thread per detected core.

4.10. General guarantees¶

Calling an entry point, or interacting with Futhark values through the functions listed above, has no system-wide side effects, such as writing to the file system, launching processes, or performing network connections. Defects in the program or Futhark compiler itself can with high probability result only in the consumption of CPU or GPU resources, or a process crash.

Using the #[unsafe] attribute with in-place updates can result in writes to arbitrary memory locations. A malicious program can likely exploit this to obtain arbitrary code execution, just as with any insecure C program. If you must run untrusted code, consider using the --safe command line option to instruct the compiler to disable #[unsafe].

Initialising a Futhark context likewise has no side effects, except if explicitly configured differently, such as by using futhark_context_config_dump_program_to(). In its default configuration, Futhark will not access the file system.

Note that for the GPU backends, the underlying API (such as CUDA or OpenCL) may perform file system operations during startup, and perhaps for caching GPU kernels in some cases. This is beyond Futhark’s control.

Violation the restrictions of consumption (see Consumption and Aliasing) can result in undefined behaviour. This does not matter for programs whose entry points do not have unique parameter types (In-place Updates).

4.11. Manifest¶

The C backends generate a machine-readable manifest in JSON format that describes the API of the compiled Futhark program. Specifically, the manifest contains:

A mapping from the name of each entry point to:
- The C function name of the entry point.
- A list of all inputs, including their type (as a name) and whether they are unique (consuming).
- A list of all outputs, including their type (as a name) and whether they are unique.
- A list of all tuning parameters that can influence the execution of this entry point. These are not necessarily unique to the entry point.
A mapping from the name of each non-scalar type to:
- The C type used to represent this type (which is in practice always a pointer of some kind).
- What kind of type this is - either an array or an opaque.
- For arrays, the element type and rank.
- A mapping from operations to the names of the C functions that implement the operations for the type. The types of the C functions are as documented above. The following operations are listed:
  - For arrays: free, shape, values, new.
  - For opaques: free, store, restore.
- For opaques that are actually records (including tuples):
  - The list of fields, including their type and a projection function. The field ordering here is the one expected by the new function.
  - The name of the C new function for creating a record from field values.

Manifests are defined by the following JSON Schema:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "https://futhark-lang.org/manifest.schema.json",
    "title": "Futhark C Manifest",
    "description": "The C API presented by a compiled Futhark program",
    "type": "object",
    "properties": {
        "backend": {"type": "string"},
        "version": {"type": "string"},
        "entry_points": {
            "type": "object",
            "additionalProperties": {
                "type": "object",
                "properties": {
                    "cfun": {"type": "string"},
                    "tuning_params": {
                        "type": "array",
                        "items": {
                            "type": "string"
                        }
                    },
                    "outputs": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "type": {"type": "string"},
                                "unique": {"type": "boolean"}
                            },
                            "additionalProperties": false
                        }
                    },
                    "inputs": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "name": {"type": "string"},
                                "type": {"type": "string"},
                                "unique": {"type": "boolean"}
                            },
                            "additionalProperties": false
                        }
                    }
                }
            }
        },
        "types": {
            "type": "object",
            "additionalProperties": {
                "oneOf": [
                    { "type": "object",
                      "properties": {
                          "kind": {"const": "opaque"},
                          "ctype": {"type": "string"},
                          "ops": {
                              "type": "object",
                              "properties": {
                                  "free": {"type": "string"},
                                  "store": {"type": "string"},
                                  "restore": {"type": "string"}
                              },
                              "additionalProperties": false
                          },
                          "record": {
                              "type": "object",
                              "properties": {
                                  "new": {"type": "string"},
                                  "fields": {
                                      "type": "array",
                                      "items": {
                                          "type": "object",
                                          "properties": {
                                              "name": {"type": "string"},
                                              "type": {"type": "string"},
                                              "project": {"type": "string"}
                                          }
                                      }
                                  }
                              },
                              "additionalProperties": false
                          }
                      },
                      "required": [ "kind", "ctype", "ops" ]
                    },
                    { "type": "object",
                      "properties": {
                          "kind": {"const": "array"},
                          "ctype": {"type": "string"},
                          "rank": {"type": "integer"},
                          "elemtype": {
                              "enum":
                              ["i8", "i16", "i32", "i64",
                               "u8", "u16", "u32", "u64",
                               "f16", "f32", "f64",
                               "bool"]
                          },
                          "ops": {
                              "type": "object",
                              "properties": {
                                  "free": {"type": "string"},
                                  "shape": {"type": "string"},
                                  "values": {"type": "string"},
                                  "new": {"type": "string"}
                              },
                              "additionalProperties": false
                          }
                      }
                    }]
            }
        }
    },
    "required": ["backend", "entry_points", "types"],
    "additionalProperties": false
}

It is likely that we will add more fields in the future, but it is unlikely that we will remove any.