Tutorials
Last updated
Was this helpful?
Last updated
Was this helpful?
To know more about of Gen with Julia you can find more information in the following link
In Gen, probabilistic models (both generative models and conditional inference models) are represented as generative functions. Gen provides a built-in modeling language for defining generative functions (Gen can also be extended to support other modeling languages, but this is not covered in this tutorial). This tutorial introduces the basics of Gen’s built-in modeling language, and illustrates a few types of modeling flexibility afforded by the language, including:
Using a stochastic branching and function abstraction to express uncertainty about which of multiple models is appropriate.
Representing models with an unbounded number of parameters (a ‘Bayesian non-parametric’ model).
Gen is a package for the Julia language. The package can be loaded with:
Gen programs typically consist of a combination of (i) probabilistic models written in modeling languages and (ii) inference programs written in regular Julia code. Gen provides a built-in modeling language that is itself based on Julia.
This tutorial uses a Jupyter notebook. All cells in the notebook are regular Julia cells. In Julia, semicolons are optional at the end of statements; we will use them at the end of some cells so that the value of the cell is not printed.
This notebook uses the PyPlot Julia package for plotting. PyPlot wraps the matplotlib Python package.
This notebook will make use of Julia symbols. Note that a Julia symbol is different from a Julia string:
Probabilistic models are represented in Gen as generative functions. The simplest way to construct a generative function is by using the built-in modeling DSL. Generative functions written in the built-in modeling DSL are based on Julia function definition syntax, but are prefixed with the @gen
keyword. The function represents the data-generating process we are modeling: each random choice it makes can be thought of as a random variable in the model. The generative function below represents a probabilistic model of a linear relationship in the x-y plane. Given a set of x coordinates, it randomly chooses a line in the plane and generates corresponding y coordinates so that each (x,y)
is near the line. We might think of this function as modeling house prices as a function of square footage, or the measured volume of a gas as a function of its measured temperature.
The generative function takes as an argument a vector of x-coordinates. We create one below:
Given this vector, the generative function samples a random choice representing the slope of a line from a normal distribution with mean 0 and standard deviation 1, and a random choice representing the intercept of a line from a normal distribution with mean 0 and standard deviation 2. In Bayesian statistics terms, these distributions are the prior distributions of the slope and intercept respectively. Then, the function samples values for the y-coordinates corresponding to each of the provided x-coordinates.
This generative function returns the number of data points. We can run the function like we run a regular Julia function:
More interesting than n
are the values of the random choies that line_model
makes. Crucially, each random choice is annotated with a unique address. A random choice is assigned an address using the @trace
keyword. Addresses can be any Julia value. In this program, there are two types of addresses used – Julia symbols and tuples of symbols and integers. Note that within the for
loop, the same line of code is executed multiple times, but each time, the random choice it makes is given a distinct address.
Although the random choices are not included in the return value, they are included in the execution trace of the generative function. We can run the generative function and obtain its trace using the simulate
method from the Gen API:
This method takes the function to be executed, and a tuple of arguments to the function, and returns a trace and a second value that we will not be using in this tutorial. When we print the trace, we see that it is a complex data structure.
A trace of a generative function contains various information about an execution of the function. For example, it contains the arguments on which the function was run, which are available with the API method get_args
:
The trace also contains the value of the random choices, stored in map from address to value called a choice map. This map is available through the API method get_choices
:
We can pull out individual values from this map using Julia’s subscripting syntax [...]
:
We can also read the value of a random choice directly from the trace, without having to use get_choices
first:
The return value is also recorded in the trace, and is accessible with the get_retval
API method:
In order to understand the probabilistic behavior of a generative function, it is helpful to be able to visualize its traces. Below, we define a function that uses PyPlot to render a trace of the generative function above. The rendering shows the x-y data points and the line that is represented by the slope and intercept choices.
Because a generative function is stochastic, we need to visualize many runs in order to understand its behavior. The cell below renders a grid of traces.
Now, we generate several traces and render them in a grid