E101 | E102 | E103 | E104 | E105 | E106 | E107 | E108 | E109 | E110 | E111 | E112 | E113 | E114 | E115 | E116 | E201 | E202 | E203 | E204 | E205 | E206 | E207 | E208 | E209 | E210 | E211 | E212 | E213 | E214 | E215 | E216 | E217 | E218 | E219 | E220 | E221 | E222 | E223 | E224 | E225

General syntax | Variables | Continuous time models | Basic operations | Conditionals | Operators | Mathematical functions and constants | Arrays | Array size | Special functions for arrays | Distribution functions | Semantics of random number draws | Special functions | Parameters | Data | Interpolation | Restricted names

Order of events | Options | odin2.compatibility | odin2.check_bounds | odin2.target

Rebuilding the bootstrap library | Testing a copy of hipercow on the cluster | Recreating the vignettes | Rtools, Java support and R versions (Windows) | Adding R Versions | Updating RTools | Updating Java support | stan

Using cyphr

Type of location | An example | Possible working patterns | Pulling complete trees, or not | Moving as little data as possible | Interaction between source tree and .outpack | Sharing packets with collaborators using a shared file system

The basic idea | An example | Create a tiny package | Handle the configuration | Evaluate the query | Trying it out | Making the plugin more robust | Saving metadata about what the plugin did | Potential uses

Using dependencies | Basic use | Filtering candidates by parameters | Interpreting errors | Filtering candidates in other ways | Computing dependencies and using many dependencies at once | How dependencies interact with locations | Other points

orderly.quiet | orderly.index_progress | orderly.schema_validate | orderly.disable_orderly2_compat | orderly.git_error_ignore | orderly.git_error_is_warning | orderly.interactive_parameters_missing_error | Environment variables

Installation | Creating an empty orderly repository | Creating your first orderly report | Depending on packets from another report | Available in-report orderly commands | Parameterised reports | Shared resources | Strict mode | Interactive development | Deleting things from the archive | Interaction with version control | Interaction with the outpack store | Relationship between orderly and outpack

Summary of changes | So long YAML and thanks for all the whitespace errors | Implications | Database support has been moved into a plugin | No more commit | No more testing or development mode | New, language-agnostic, backend | Other, smaller changes | What is missing compared with orderly1 | How to migrate | What about the packages? | What were the problems in version 1 | Working with old versions without migrating | Users of orderly2 | Migrate your source code | Use the built-in compatibility support | Install the orderly2 compatibility package

Collaborative analysis | Reproducible analyses | Difference from a workflow system | What is reproducibility anyway? | I'm in, where do I start?

Structure of queries | Special simple queries | Scoping queries | Scoping on name | Dependencies | Possible future queries and interface improvements | Simple things | Explain the query

Basic overview | What goes into a packet | Types of users of outpack | Directory layout | Configuration (.outpack/config.json) | Packet metadata (.outpack/metadata/) | Location information (.outpack/location/) | A file store (.outpack/files) | An archive (archive/ by default) | Adding a packet | Adding metadata | Adding files | Marking the packet as unpacked and known locally | Ordering of operations | Details | Outpack ids | Location ids | Representing hashes | Times | File store

Outpack files accidentally committed to git | I don't care about my git history at all | I just want this to go away and nothing I have committed is very large | I care about my history but want this stuff gone

Configuration | Use within a report | More on configuration | Environment variables for passwords | Advanced database configuration | Migrating from orderly v1 | Other differences in behaviour

Discrete time stochastic SIR model

E0001 | E1001 | E1002 | E1003 | E1004 | E1005 | E1006 | E1007 | E1008 | E1009 | E1010 | E1012 | E1013 | E1014 | E1015 | E1016 | E1017 | E1018 | E1019 | E1020 | E1021 | E1022 | E1023 | E1024 | E1025 | E1026 | E1027 | E1028 | E1029 | E1030 | E1031 | E1032 | E1033 | E1035 | E1036 | E1037 | E1038 | E1039 | E1040 | E1041 | E1042 | E1043 | E1044 | E1045 | E1046 | E1047 | E1048 | E1049 | E1050 | E1051 | E1052 | E1053 | E1054 | E1055 | E1056 | E1057 | E1058 | E1059 | E1060 | E1061 | E1062 | E1063 | E1064 | E1065 | E1066 | E1067 | E1068 | E1069 | E1070 | E2001 | E2002 | E2003 | E2004 | E2005 | E2006 | E2007 | E2008 | E2009 | E2010 | E2011 | E2012 | E2013 | E2014 | E2015 | E2016 | E2017 | E2018 | E2019 | E2020 | E2021 | E2023 | E2024 | E2025 | E2026 | E2027 | E2028 | E2029 | E3001 | E3002 | E3003

Serialisation | Pipelining | Subscriptions | Using commands that are not yet supported | Potential applications | Scripts | Getting help

When might you want to create a new sampler? | A toy problem | A toy sampler

Introduction | The problem | The process | Example | Running the report | Creating a report | Resources, sources and artefacts | Using artefacts from other reports | Parameterised reports | Using global resources | Using version control | Using SQL databases | Configuration | Use within a report | Advanced use | Customising the database configuration | Advanced database configuration | Using environment variables and secrets | Developing a report

Overview | The process | Limitations

The orderly changelog | User's perspective | Details

Individual researcher | Why not commit the archive to git? | Small group, sharing via shared disk | Some comments on collaboration | Group of researchers, sharing via server | Using staging environments | Dependency resolution | Looking forward, a fully decentralised approach

OrderlyWeb | Configuring orderly to talk to OrderlyWeb | Interacting with the remote server | Future developments | Deploying a remote server

Setting the scene | Comparing to data | Stochastic likelihood with a particle filter | Inference with particle MCMC (pMCMC) | Effective sampling | Comparing against vectors of data | Deterministic models from stochastic | Differentiable models | Running multiple parameter sets at once | For dust2 systems | For dust2 filters/unfilters | For monty models | Further reading

New features | Planned | Hoped | Missing features | Changes in syntax | user() becomes parameter() | Compare keyword is now removed. | Vector parameters assign without array indices | Vector/matrix/array parameters whose size is determined by input require rank argument | Interpolate results assign without array indices | Discrete-time models have a more solid time basis | Assignments to dt | Assignments to time | Use of step | The name of the time variable in continuous time models has changed | Random number function calls have changed | System size cannot be changed after creation | Changes in the way arrays are handled | Changes to how delays are supported | delay() cannot be used in discrete time models | delay() results assign without array indices | The first argument to delay() must be the name of an equation | The 'default' argument to delay() has been removed | The delay time must be constant | General changes | Known limitations | Much slower compilation time | Loss of features from odin 1.x | Updating old code | Avoiding output() in discrete-time systems

First steps, a random walk | Support code | Annotations | Type definitions | The methods | Multiple variables, the SIR revisited | Continuous time (ODE) models | Continuous time models with additional output | Continuous time models with special variables

Options | Influential options | hipercow.auto_install_missing_packages | hipercow.progress | hipercow.timeout | hipercow.validate_globals | hipercow.max_size_local | hipercow.default_envvars | hipercow.development | DIDE options | rrq options | hipercow.rrq_offload_threshold_size | Options from other packages | Setting options | R versions | Workflow considerations | Long running tasks | Disk space

Pre-requisites:- | Authentication with DIDE | About our usernames and passwords | Networks | Filesystems and paths | Cluster-based storage and home directories | Mapping network drives on your computer | Windows | macOS | Linux | Initialisation | Does it work? | Default environment variables | The Nodes | Cluster Storage | Cluster Storage on Windows Compute nodes | Cluster Storage on Linux Compute nodes

Basics | Loading packages | Loading your own functions | Defining globals | Other points | Relationship with provisioning

Clusters and Platforms | Installing prerequisites | Filesystems and paths | Initialising | Running your first task | Using functions you have written | Getting information about tasks | Fetching information with task_info() | Fetching logs with task_log_show | Watching logs with task_log_watch | Running many tasks at once | Bulk call, or "parallel map" | Bulk expression | More on bundles | Picking bundles back up again later | Making bundles from tasks | Parallel tasks | Understanding where variables come from | Cancelling tasks | Retrying tasks

Differences | Mapping of didehpc methods to hipercow functions | queue_didehpc | task | bundle | Configuration | Missing features | New features

Using pkgdepends | A list of packages | Manually | Automatically, from an environment | Using a script | Using renv | Some details about the process

Task-level parallelism | What resources does the cluster have? | Specifying multi-core resources | Running parallel tasks | Using the Parallel package | Using the future package | Specifying more work than there are cores | How many cores should each process use? | Multiple cores per process | Other ways of using cores | Specifying which nodes should run your tasks | Memory requests | Running on specific nodes | Selecting by queue | Selecting by node names | Task time limits and scheduling | The maximum runtime | Delaying tasks starting | Lowering your priority

My task failed | Caused by an error in your code | Caused by an error during startup | My task got stuck at submitted | My code works on my computer but not on the cluster | Some of my tasks work on the cluster, but others fail | My code is slower on the cluster than running locally! | I can't connect to the cluster | Asking for help

Local installations | Latest available version of INLA for your R version | Specific version of INLA, where available. | Hipercow provisioning | Latest version of INLA for latest R version. | Latest version of INLA for older R version

Local installations | CmdStan on the cluster | Installation and versions | Compiling models | A worked example

Getting started | The lightweight queue pattern | Basic usage | Scaling up | Interprocess commuication pattern | Tricks and tips | Controlling the worker environment | General considerations | Stopping redundant workers | Permanence | Object storage

Memoisation

Key-value store | Namespaces | Bulk get/set | Import / export | Supported backends | Implementation details | Content addressable lookup | Environment-based caching | Classed exceptions

Combining schemas

Where is everything? | The particle filter | Other state space methods | Running MCMC

Some simple examples | Calculations in the DSL | Pass in fixed data

Comparisons | The bendy banana | Sampling with other samplers

The structure of monty samples | Conversion to posterior's draw objects | Conversion to coda's mcmc.list objects

Dude, where's my feature? | Change in meaning of time in discrete-time models | New interface | See also

Error handling | Increasing resilience via separate processes | Loss of workers | Retrying tasks | An example where retrying fixes an error | Performance considerations | Deleting tasks that have been retried | Going further

Messages and responses | PING | ECHO | INFO | EVAL | PAUSE / RESUME | SET_TIMEOUT / GET_TIMEOUT | Messages that are supported but use via wrappers:

Getting started | Hello world | Design | Running tasks | Configuring the worker environment | Scheduling options | Tasks that depend on other tasks | Multiple queues | Running tasks in separate processes | Coping with memory use | Orchestrating workers | Use (and limitations) of spawn | Start workers on another node, perhaps using a scheduler | Use docker | Waiting for workers to appear | Worker heartbeat | Getting a Redis server | Install Redis | Use Redis on a different machine

Using print() | print format strings | Conditional display | Controlling precision | Current limitations | Show the generated code | Use the interactive debugger

A simple example - random walk | Running a model in parallel | A more interesting example | Other methods | Reordering particles | Set particle state

Influential environment variables and options

Running multiple realisations | Parallelisation | Efficient running | Efficient state handling | Useful verbs | A compilation target

The basic idea | An example

Discrete time systems | Continuous time systems | Properties of resettable variables | Unresolved bits that might change

Handling lack of vault gracefully | Installing vault

Connecting to vault | Reading, writing, listing and deleting secrets | Alternative login approaches | Username and password (userpass) | GitHub (github) | LDAP (ldap)

Avoiding output() | Limited support for interpolate() | Delays are not supported | Not all distributions are supported | Interface differences

Using MCMC to infer parameters

The packages | Documentation | The TypeScript ecosystem | Roadmap

Coping with missing data

Single variable: Logistic growth | Specifying parameters | More than one variable: the Lorenz attractor | Delay models | Arrays | Generalised Lotka-Volterra model | Interpolating functions

Ordinary differential equation models | Models implemented in R | Dense output and interpolation | Delay differential equation models | A model in R | Differences with deSolve | Models implemented in C

A simple example - random walk | Model code | Constructing a model | Running a model in parallel | A more interesting example | Other methods | Reordering particles | Set particle state | Reset the model | Use within a package

Background using R's random number generator | Basic implementation using dust | Parallel implementation with dust and OpenMP | More on the pointer object

Running multiple realisations | Parallelisation | Efficient running | Efficient state handling | Useful verbs | A compilation target

Using print() | print format strings | Conditional display | Controlling precision | Current limitations

Setup | All-in-one | Restarting

Stochastic SIR model definition | Inferring parameters with mcstate | Model data | Defining the comparison function | Inferring parameters | Using MCMC to infer parameters | Tuning the pMCMC | Running predictions | Fitting to multiple datastreams

Stochastic SIR model definition | Implementing the SIR model using odin.dust | Saving a model into a package | Running the SIR model with dust | Adding age structure to the model

A note on seeding | Distributed seeding | Continuing the streams | Considerations | Use cases | Summary

Principles | Running a model with GPU support | Writing a GPU-capable model | Data comparison functions | Developing a GPU model

Supported distributions | Performance | Underlying random number engine | Reusing the random random number generator in other projects | In a package | Standalone, parallel with OpenMP | Standalone, parallel on a GPU | Other packages with similar functionality

Discrete compartmental models in a nutshell | From continuous to discrete time | Stochastic processes | Binomial distribution | Poisson distribution | Multinomial distribution | Implementation using odin | Deterministic SIR model | Stochastic SIR model | A stochastic SEIRDS model

Basic operators | Array support | Operators | Mathematical operators | Stochastic models

Setting up an IF2 run | Running the IF2 algorithm

Model data | Comparison, model and particle filter | Nested parameters | pMCMC and Visualisations

Within-model parallelism | Between-chain parallelism | Considerations

Random bytes | UUIDs | Adjective animal | Random sentences | proquints | Roll your own identifiers | Relationship to random number generation

Box-Muller | Polar | Ziggurat | Sampling | Sampling from the tail | The edges | Optimisations

Containers | Working with containers | Run commands in running containers with exec | Copy files into and out of containers | Images | Pulling | The object returned by pull is an image object - that can be | created by using $get | Building | Importing | Working with image objects | Volumes | Working with volume objects | Networks | Working with network objects | Other docker functions

Considerations

Introduction | The environment | Transactions | Simple operations (put, get, del, etc) | Multiple transactions at once | Non-string data | Caveats

Keys and the like | Symmetric encryption | openssl | sodium | Asymmetric encryption ("key pairs") | Encrypting things | Objects | Strings | Plain raw data | Files | An even higher level interface for files | Session keys | Further reading

Minimal example | Details & disclosure | Limitations

Start a container with ports published to the host

Using a database in testing | Testing shiny apps | Package testing

Identify the Montagu Server | Basic Information | Touchstones | Scenarios | Expectations - years, ages, countries and outcomes | Demography | Gender | Format | Templates | Coverage | Long or Wide format | Returning the full range of countries | Burden Estimate Sets | Creating a new burden estimate set | Uploading data | Plotting previous burden estimate sets | Stochastic Runs | Model Run Parameter Sets | Appendix | Models | Demographic Sources

Package preparation | The API

Scope | Background | How datastorr works | The package interface | Worked example | Access control | Semantic versioning of data | Beyond GitHub | How it actually works