Quote:
Originally Posted by Damien
I would argue MS Access is also not up to the job but Excel certainly isn't.
There are other ways to present data including exporting into spreadsheets from a database. It sounds like here the master version of the data was in Excel and was being appended too via CSV files.
|
That apparently wasn't the problem. It wasn't a case of more than 1m rows, but more than 64K rows or 1400 cases.
Link
Quote:
The issue was caused by the way the agency brought together logs produced by the commercial firms paid to carry out swab tests for the virus.
They filed their results in the form of text-based lists, without issue.
PHE had set up an automatic process to pull this data together into Excel templates so that it could then be uploaded to a central system and made available to the NHS Test and Trace team as well as other government computer dashboards.
The problem is that the PHE developers picked an old file format to do this - known as XLS.
As a consequence, each template could handle only about 65,000 rows of data rather than the one million-plus rows that Excel is actually capable of.
And since each test result created several rows of data, in practice it meant that each template was limited to about 1,400 cases. When that total was reached, further cases were simply left off.
Until last week, there were not enough test results being generated by private labs for this to have been a problem - PHE is confident that test results were not previously missed because of this issue.
|