Solution to the bad query problem

Let's try to solve the bad query problem.

Obviously if we allow the application to call getData() with arbitrary $[t_{start}\ldots t_{end}]$ time intervals, we have a problem because they can easily overlap the validity intervals stored into the database.

We have to restrict what t_start and t_end the application can use.

What restrictions make sense? We have already decided that getData() is called once for each parameter at the beginning of the run to get the values to be used in the analysis of that run.

Let us demand that t_start and tend of any query be the run start and end times. Or at least that

$$start_{run} \leq start_{query} < end_{query} \leq end_{run}$$

Does it help? Yes, but not quite. We still get into trouble if data in the database changes in the middle of a run, as in Figure 8.

  

Figure 9:

Here we have eliminated the bad queries similar to query₂and were doing fine until we hit query₃, which is bad because cal₄ ends in the middle of run₄.

Bad query₃ cannot be fixed by putting restrictions on the type of queries that aplications are allowed to make. We need to apply restrictions on the data in the database.

Let's demand that validity intervals also always match run start and end times, as in Figure 9

  

Figure 10:

In this example I added a quirk -- the `special Ian's' calibration cal₅ which spans multiple runs and overlaps with cal₃.

But even with this quirk, the restriction that queries are always done on run boundaries (or inside), makes it impossible to do bad queries. Any possible query is well behaved.

Conclusion: the bad query problem is solved by requiring that validity intervals stored in the database correspond to the run start and end times; and that queries are always done on one run at a time, so that $start_{run} \leq start_{query} < end_{query} \leq end_{run}$.