05-10-2020, 10:59
|
#6226
|
cf.mega poster
Join Date: Dec 2013
Posts: 14,240
|
Re: Coronavirus
More info on those data issues.
Quote:
A technical glitch which meant almost 16,000 Covid-19 cases went unreported has been slammed as "shambolic" after it emerged that it led to delays in tracing contacts of people with coronavirus. The number of UK Covid cases surged by 22,961 on Sunday after it was revealed that thousands of cases were not included in daily reports due to some data files reporting positive test results exceeding the maximum file size.
|
https://www.msn.com/en-gb/news/uknew...rtan-dhp-feeds
|
|
|
05-10-2020, 13:09
|
#6227
|
Sulking in the Corner
Join Date: Jul 2009
Location: RG41
Services: 1 Gbps; Hub 4 MM; ASUS RT-AX88U; Ultimate VOLT. BT Infinity2; Devolo 1200AV
Posts: 11,955
|
Re: Coronavirus
Quote:
Originally Posted by Pierre
I wouldn't rush to it either. Vaccinate the at risk groups as a priority anyway. The vast majority of the population don't need the vaccine anyway.
|
My current thinking is that I wouldn't touch the vaccine.
I'm afraid of the auto-immune system effect. I get flu if I take the flu jab (so I've stopped doing that). I've covered this a few pages ago.
---------- Post added at 11:17 ---------- Previous post was at 11:08 ----------
Quote:
Originally Posted by 1andrew1
|
Quote:
A technical glitch which meant almost 16,000 Covid-19 cases went unreported has been slammed as "shambolic" after it emerged that it led to delays in tracing contacts of people with coronavirus. The number of UK Covid cases surged by 22,961 on Sunday after it was revealed that thousands of cases were not included in daily reports due to some data files reporting positive test results exceeding the maximum file size.
|
How can a file exceed the maximum file size (oxymoron)? Unless the receiving software has a maximum file size constraint. And how would that happen? Unless they were on 16 bit software it would have to have been a deliberately introduced constraint.
Far more likely that the receiving software didn't have dynamic table sizes.
Obviously there is a design problem with the back-end software but if they don't employ me to validate all this, what can you expect!!
---------- Post added at 13:09 ---------- Previous post was at 11:17 ----------
Quote:
Originally Posted by Sephiroth
My current thinking is that I wouldn't touch the vaccine.
I'm afraid of the auto-immune system effect. I get flu if I take the flu jab (so I've stopped doing that). I've covered this a few pages ago.
---------- Post added at 11:17 ---------- Previous post was at 11:08 ----------
How can a file exceed the maximum file size (oxymoron)? Unless the receiving software has a maximum file size constraint. And how would that happen? Unless they were on 16 bit software it would have to have been a deliberately introduced constraint.
Far more likely that the receiving software didn't have dynamic table sizes.
Obviously there is a design problem with the back-end software but if they don't employ me to validate all this, what can you expect!!
|
Further to the above, the PM said on TV that the "data had been truncated".
This would mean that the back-end software did not mitigate the failure mode of a system file size limit, which is c. 8 PetaBytes in W10. However, I've seen hospitals in my local trust still using Vista where the maximum is 2TB (on 32 bits).
__________________
Seph.
My advice is at your risk.
|
|
|
05-10-2020, 13:18
|
#6228
|
Remoaner
Cable Forum Team
Join Date: Mar 2004
Posts: 32,237
|
Re: Coronavirus
Appranrtly they had the database in Excel. Not sure if that means they had some of interface on top of Excel or where literally just using Excel itself.
|
|
|
05-10-2020, 14:14
|
#6229
|
The Invisible Woman
Cable Forum Team
Join Date: Jun 2003
Location: between Portsmouth and Southampton.
Age: 71
Services: VM XL TV,50 MB VM BB,VM landline, Tivo
Posts: 40,165
|
Re: Coronavirus
If only we could get a track and trace system going.
__________________
Hell is empty and all the devils are here. Shakespeare..
|
|
|
05-10-2020, 14:35
|
#6230
|
vox populi vox dei
Join Date: Mar 2007
Location: the last resort
Services: every thing
Posts: 13,739
|
Re: Coronavirus
Quote:
Originally Posted by Maggy
If only we could get a track and trace system going.
|
We have a fine tradition of making a complete hash of anything to do with data in this country.
__________________
To be or not to be, woke is the question Whether 'tis nobler in the mind to suffer. The slings and arrows of outrageous wokedome, Or to take arms against a sea of wokies. And by opposing end them.
|
|
|
05-10-2020, 14:48
|
#6231
|
cf.mega poster
Join Date: Dec 2013
Posts: 14,240
|
Re: Coronavirus
Quote:
Originally Posted by papa smurf
We have a fine tradition of making a complete hash of anything to do with data in this country.
|
Which country might that be?
|
|
|
05-10-2020, 14:50
|
#6232
|
vox populi vox dei
Join Date: Mar 2007
Location: the last resort
Services: every thing
Posts: 13,739
|
Re: Coronavirus
Quote:
Originally Posted by 1andrew1
Which country might that be?
|
England mostly.
__________________
To be or not to be, woke is the question Whether 'tis nobler in the mind to suffer. The slings and arrows of outrageous wokedome, Or to take arms against a sea of wokies. And by opposing end them.
|
|
|
05-10-2020, 16:01
|
#6233
|
laeva recumbens anguis
Cable Forum Team
Join Date: Jun 2006
Age: 67
Services: Premiere Collection
Posts: 42,136
|
Re: Coronavirus
Quote:
Originally Posted by Damien
Appranrtly they had the database in Excel. Not sure if that means they had some of interface on top of Excel or where literally just using Excel itself.
|
What the whatting what?
Quote:
The problems are believed to have arisen when labs sent in their results using CSV files, which have no limits on size. But PHE then imported the results into Excel, where documents have a limit of just over a million lines.
The technical issue has now been resolved by splitting the Excel files into batches.
|
https://www.dailymail.co.uk/news/art...el-glitch.html
Glitch, my arse - who in (insert appropriate diety’s here) name uses a million+ line spreadsheet, besides complete amateurs? I spent most of the last 20 years getting home-grown systems off spreadsheets and migrated to ‘proper’ IT systems, because the spreadsheets tend to have no documentation, ownership, or resilience.
This is NOT an IT glitch. It’s using a spreadsheet to do the job of a database. It’s not fit for purpose. It’s trying to hammer a nail in with a roll of sandpaper; the tool doesn’t suit the job.
__________________
There is always light.
If only we’re brave enough to see it.
If only we’re brave enough to be it.
If my post is in bold and this colour, it's a Moderator Request.
Last edited by Hugh; 05-10-2020 at 16:05.
|
|
|
05-10-2020, 16:25
|
#6234
|
cf.mega poster
Join Date: Dec 2013
Posts: 14,240
|
Re: Coronavirus
Quote:
Originally Posted by Hugh
What the whatting what?
https://www.dailymail.co.uk/news/art...el-glitch.html
Glitch, my arse - who in (insert appropriate diety’s here) name uses a million+ line spreadsheet, besides complete amateurs? I spent most of the last 20 years getting home-grown systems off spreadsheets and migrated to ‘proper’ IT systems, because the spreadsheets tend to have no documentation, ownership, or resilience.
This is NOT an IT glitch. It’s using a spreadsheet to do the job of a database. It’s not fit for purpose. It’s trying to hammer a nail in with a roll of sandpaper; the tool doesn’t suit the job.
|
It's more World-leading cut and paste than World-Leading Track and Trace.
|
|
|
05-10-2020, 18:00
|
#6235
|
cf.mega poster
Join Date: Apr 2004
Location: Northampton
Services: Virgin Media TV&BB 350Mb,
V6 STB
Posts: 7,866
|
Re: Coronavirus
It would've had to be a spreadsheet rather than a database, simply because people are much more familiar with spreadsheets than databases.
MS Access isn't a standard part of MS Office.
Sounds like the data was originally in a database, but had to be sent out in a more user friendly format.
|
|
|
05-10-2020, 18:16
|
#6236
|
Remoaner
Cable Forum Team
Join Date: Mar 2004
Posts: 32,237
|
Re: Coronavirus
Quote:
Originally Posted by nomadking
It would've had to be a spreadsheet rather than a database, simply because people are much more familiar with spreadsheets than databases.
MS Access isn't a standard part of MS Office.
Sounds like the data was originally in a database, but had to be sent out in a more user friendly format.
|
I would argue MS Access is also not up to the job but Excel certainly isn't.
There are other ways to present data including exporting into spreadsheets from a database. It sounds like here the master version of the data was in Excel and was being appended too via CSV files.
|
|
|
05-10-2020, 18:43
|
#6237
|
cf.mega poster
Join Date: Apr 2004
Location: Northampton
Services: Virgin Media TV&BB 350Mb,
V6 STB
Posts: 7,866
|
Re: Coronavirus
Quote:
Originally Posted by Damien
I would argue MS Access is also not up to the job but Excel certainly isn't.
There are other ways to present data including exporting into spreadsheets from a database. It sounds like here the master version of the data was in Excel and was being appended too via CSV files.
|
That apparently wasn't the problem. It wasn't a case of more than 1m rows, but more than 64K rows or 1400 cases.
Link
Quote:
The issue was caused by the way the agency brought together logs produced by the commercial firms paid to carry out swab tests for the virus.
They filed their results in the form of text-based lists, without issue.
PHE had set up an automatic process to pull this data together into Excel templates so that it could then be uploaded to a central system and made available to the NHS Test and Trace team as well as other government computer dashboards.
The problem is that the PHE developers picked an old file format to do this - known as XLS.
As a consequence, each template could handle only about 65,000 rows of data rather than the one million-plus rows that Excel is actually capable of.
And since each test result created several rows of data, in practice it meant that each template was limited to about 1,400 cases. When that total was reached, further cases were simply left off.
Until last week, there were not enough test results being generated by private labs for this to have been a problem - PHE is confident that test results were not previously missed because of this issue.
|
|
|
|
05-10-2020, 18:45
|
#6238
|
cf.mega poster
Join Date: Jun 2003
Age: 37
Services: Plusnet FFTC
Posts: 4,822
|
Re: Coronavirus
Dominic Cummings is particularly keen on filling the civil service with physicists. I can tell you mistakes like that would never happen. Although spelling mistakes and delays may go up.
__________________
"Knowledge is Power. Power Corrupts. Study Hard. Be Evil."
|
|
|
05-10-2020, 18:49
|
#6239
|
cf.mega poster
Join Date: Apr 2004
Location: Northampton
Services: Virgin Media TV&BB 350Mb,
V6 STB
Posts: 7,866
|
Re: Coronavirus
Quote:
Originally Posted by downquark1
Dominic Cummings is particularly keen on filling the civil service with physicists. I can tell you mistakes like that would never happen. Although spelling mistakes and delays may go up.
|
Usual nonsense. As if he would've had any say in hiring developers for PHE. They were possibly hired before he came onto the "scene".
Too much hiring on the basis of bits of paper, rather than core underlying ability.
|
|
|
05-10-2020, 18:52
|
#6240
|
cf.mega poster
Join Date: Jun 2003
Age: 37
Services: Plusnet FFTC
Posts: 4,822
|
Re: Coronavirus
Quote:
Originally Posted by nomadking
Usual nonsense. As if he would've had any say in hiring developers for PHE. They were possibly hired before he came onto the "scene".
Too much hiring on the basis of bits of paper, rather than core underlying ability.
|
I didn't say he did anything. I am saying what he would have done if he was in charge.
__________________
"Knowledge is Power. Power Corrupts. Study Hard. Be Evil."
|
|
|
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
|
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT +1. The time now is 19:20.
|