Datasets

Main link http://algaebiogas.eu/raw/data/show.php?tbl=*.

These pages contain links to:

• most of the data collected in the SCADA system of the demonstration centre,
• most of the data from manual logs,
• most of the analysis results performed in house
• most the analysis data outsourced to external organizations.

All above data is collected in a uniform database and it may be browsed by tools that are purposely made simple and without much of the presentation frills to make them useful as data sources for external data analysis and visualisation software. For this reason the data pages do not contain any otherwise standard graphical elements. Most of the data pages contain a very simple form for data selection (e.g. from date and to date), minimal meta-data (describing the meaning of the data table), pure data table, and a minimal disclaimer and legal note.

Some data may be presented in graphical form using Google charts, but more sophisticated presentation is left to other (external) tools.

Data form SCADA system is transferred daily with some data compression. Examples: some data in some periods is/was collected in half second interval, some data is collected as less reliable (flags), etc. In general the data interval for the logs is 60 seconds unless source data was collected with lower frequency.

Data from SCADA system was not cleared of invalid values if they were not flagged so in the SCADA system itself. Such values come for example from sensor cleaning or sensor calibration operations, from dry run or from other events. Interpreattion of these values is only possible with their matching to other logs, which is done at the higher levels.

SCADA logs contain a lot of internal data that has no meaningful interpretation outside of the control system. Examples of such data are debugging measurements for some sensors (e.g. sedimenter overflow), raw conductivity data, raw PID controller values for some control loops, etc. Such data is transferred to the logs, but we did not make any effort to explain their semantics as it it highly connected to the internals of the control system itself. These datasets are generally not shown on most of the lists, if really needed, you may try adding string "&Regular=*" or "&Regular=0" or to include them in the lists.

Data browser interprets all means of parameter passing in the same way (parameters specified in the URL or in the POST request and even in cookies). Internally it may use POST or GET requests (configuration option), but URI parameters may always be specified in the very same way. This enables use of the web query mechanism (e.g. in excel) in a user friendly way in the URI). Example: to get a table of pH values for the main pond with daily resolution from 1-May-2016 to 31-May-2016 you may select these in the form at http://algaebiogas.eu/raw/data/show.php?tbl=tag_day or used directly as url http://algaebiogas.eu/raw/data/show.php?tbl=tag_day&Tag=98&bDat=2016-05-01&eDat=2016-06-01.

Additional notes:

• http and https queries are equivalent
• all parameter names (and values) are case sensitive
• tbl parameter defines the main operation
• tag parameter defins the selected data item, if not specified or illegal, only selection form is shown,
• there is no caching on the server side, and caching on the client side is disabled with meta data in html head (but this is client setting that may be overridden) - all queries should go directly to the database
• all dates and date-time values are specified in ISO8601 format "YYYY-MM-DD hh:mm:ss"; some tolerance in input formats exist, but any other specification is unreliable
• default value for time if only date is specified is 00:00:00, no effort is made to convert upper limits to "23:59:59" as it might be expected - example: to get all values for a day, specify its date and bDat and next day as eDat