This is the Blog of the technical experts at GEN and its companies

USB Flash - Built in failure


With the slow decline in CD's and the long lost days of floppy diskettes, USB portable storage has become common place. A memory stick, thumb drive or pen drive are common terms for the same thing, a USB mass storage device based on FLASH, and yet many people don't know that the whole technology behind FLASH storage has a very limited lifespan - this leads me on to the relatively high volume of data recovery requests we have for USB storage coming through the channel.

Flash memory is generally of two types, NAND and NOR. Both technologies allow permenant storage of data without needing a power supply. NAND requires data to be read and written in blocks called 'pages' and is by far the most common FLASH memory in use today.

FLASH memory like all memory stores data in 0's and 1's in a vast array of cells, but the method by which the data is permanently written involves pushing a charge (electrons) through an insulated layer, once through the insulator its stuck there and will remain until its pulled back through the insulator therefore changing the state.

However, this 'pushing' and 'pulling' through the insulator, known as tunnelling slowly breaks down the insulator until it fails. When an insulator fails this only effects the cell, but of course just one bit that won't switch will adversely effect the data when read back. Furthermore certain areas of the flash drive are read and written much more than other area's and these are the master directory and the File allocation tables, both of which are changed when data is read (changing last access time) and written (changing last updated time and changing allocation of storage in the file allocation table). This means that in many instances the part of the flash drive that fails first is the most important part - the part that tells us what files are stored on the drive and where they are stored.

Cheap vs Expensive

When it comes to Flash Drives, there is a real physical difference between the budget end of the market and the professional end because NAND/NOR Flash comes in many different flavours depending on its performance and expected lifespan. Often the cheapest FLASH IC's are designed for storing firmware in embedded devices where write performance is a non issue and the expected number of writes is very limited, maybe 10 writes in its entire lifetime whereas the most expensive FLASH is designed specifically for high speed  and many write cycles and this is the correct hardware for USB Flash Drives. If you can buy a 128GB Flash drive from SANDISK for £30 and a unbranded one for £5 then the lifespan and performance of your SANDISK drive will be many many times better than the unbranded one.

I guess I should also point out that some cheap unbranded USB Flash drives (or knock off Branded) are engineered to falsely report their capacity. This is done by creating a partition on the drive with false data, so the computer you connect it to thinks its larger than it is and the only way to be sure is to try and fill it up or to perform a low level reformat. This sort of storage fraud is often seen on sites like eBay promising 1TB of flash for $10 which is nonsense.

Recovering data from failed Flash drives isn't that hard, but it does bring with it some challenges because the data will have errors in it where specific cells are stuck or indeed entire pages are stuck and non responsive and its not always possible to identify these area's during the scan, they often read as ok but with incorrect data, or they read as all 0's but after re-assembling the filesystem as best we can its over to the client to work through the recovered data and validate it.

The bottom line here is never ever rely on a USB Flash drive for data storage, its not safe and certainly not guaranteed and it will fail at some point. Stick with brand names and stay away from the budget end of the market.

Continue reading
  4764 Hits
4764 Hits

Today at the Helpdesk - ITV Player

A Customer raised a ticket at the HelpDesk today complaining that their ITV player was no longer working correctly and giving strange messages. We asked for a screenshot and received it minutes later. The screen show indicated that 'Ad Block Software detected' which peaked our attention and so we investigated further. 

Now ITV Player is one of the few remaining companies still using Adobe Flash Player to stream movies despite there being much better transports available (like HTML5 etc) and flash gives us a number of problems here because (a) shockwave/flash is blocked by default at the firewall (as it is for all our customers of officeGateway), and (b) none of our workstations have flash installed as its a high security risk. Anyhow after some faffing about we managed to get the adobe flash demo page to work and then switched on over to ITV player. 

After selecting a program to watch, the usual unclean and tatty flash player window is displayed, and we click the big > in the middle to play. Immediately we're given 5 minutes of adverts to watch before we can do anything else like FF/REW etc. Then after the 5 minutes of ad's we're onto the programmes introduction for another 2 minutes and then finally the show begins. Now at this point we followed the EU's reported behaviour of fast forwarding to the second segment of the show, and doing this means that again we have to watch another 5 minutes of adverts, and then on the last advert....

Perfect, we can reproduce the issue in a freshly installed (today) system of Safari 9.1 on OSX 11 with definitely no ad block software installed. After this rather abrupt message the player is dead and you have to refresh the page and whilst the option to 'Resume' is offered it does nothing except start from the beginning again meaning... watch another 5 minutes of adverts, FF then watch another 5 minutes of adverts and finally get to the segment you need to watch, but on our second try we got something else...

And yet again, flash player is dead and we've got to reload the page, another 5 + 5 = 10 minutes of the same pointless adverts and then the show plays just fine. We shuffle back and fourth several times after this and it seems to play everything just fine, we even left it playing in the background and it got through another two episodes before the 'Ad Block Software detected' message truncated our viewing enjoyment. So, being 'flash' which is so easily reversed we downloaded the SWF file and took a look at the cause of these spurious and erroneous messages. 

Looking at the code(scripts) within the SWF file it would appear that the ad block software message is triggered when a HTTP request fails, but that wouldn't necessarily mean its ad block software would it? In our short tests today the player has shown itself to be far less than reliable on a fresh install of OSX and if all it takes is a HTTP error to cause it all to come crashing down then someone really should sort that out. Whilst we were in the code we did notice a significant level of logging and auditing taking place that I'm fairly sure no one knows is happening but that's another story for another day.

For effective Ad Blocking with this shockingly poor flash setup it would be much easier to redirect the SWF request to a crippled SWF (or decompiled/recompiled) with the ad's removed. If, on the other hand someone actually wanted to add in 'Ad blocking' software detection then doing it within HTML5 would be far simpler with some client side js/java passing a token back to a server somewhere then a reliable solution is to be had. Of course, once you've spent a few £££ on that solution then ad block software vendors will find a way around it by trashing your client side js so one has to wonder if the battle is even worth the expense? In ITV's case I strongly suspect their spurious 'Ad Block Software detected' message simply serves as a catalyst for the viewer to hit google and discover that Ad Block software does exist and how to download and install it. I suppose its a little like "thepiratebay" that virtually no one had ever heard of until some muppet decided to sue them and then suddenly the whole world knew about it and sites like it and moreover how to get around all the worthless 'blocks' that ISP's were forced to setup by clueless judges. 

So back to the ticket in question, we couldn't of course fix ITV's failures to provide a stable service but the whole idea of 'Ad Block Software' gave us another avenue to explore - Ad Block Software! We searched the internet and found several solutions all promising to remove all ad's and thought we should give them a try. I'm going to go ahead and call these Program 1, 2 and 3 and not give out the actual names of the software as I don't want to encourage anyone to install software that's not been fully certified as safe but if you have the knowledge then google/bing is your friend. 

  1. Program 1 is open source, freely available and seems to have a fairly active github repository and once installed we found it did indeed block some ad's but not ITV's. It didn't however increase the incidence of the erroneous "Ad Block Software detected" message and in fact it seems to occur less often with it installed but that may be coincidental. So we uninstalled that and moved on to...
  2. Program 2 is closed source but freely available and seems well supported. We installed it without issue and again it did stop some ad's from some 'other' websites but for ITV player it stopped it working altogether. We found that we could specify various options to make it work again but we still got ad's and the erroneous message still appeared from time to time but no more than with nothing installed. So uninstall and move on to...
  3. Program 3 is closed source and not free but we did managed to acquire a temporary licence from the vendor for our testing, this installed without issue and finally our ITV player was advert free and without any 'Ad Block Software detected' message either. With a little more investigation into the settings it was clear that this plug-in was operating at a much lower level than Program 1 & 2. 

So, in summary, the message is in error and it clearly only serves to annoy potential viewers but when you look at the whole ITV player setup, being forced to watch 10 minutes of adverts, that's 40 minutes per hour is in itself going to alienate customers especially if they are just the same ad's over and over again which is what we observed. I personally think YouTube has the balance about right (and I'm rarely one to support Google) with its skippable adverts which means that if I'm not interested then I'm not forced to watch it all, but on the other hand it means that the adverts that do interest me I can watch in full and I do watch some in full just in case anyone wondered. 

This article is a technical article and the content is solely the opinion of the author and not the company. E&OE. Neither the author nor the company has any interest in ITV, its player or any solution designed to remove advertising and does not recommend you do or do not install any such solution. 

Follow up

So, the same customer contacted us again today via the HelpDesk to tell us that ITV Player was now not working at all and instead was saying "ITV Hub is only available to viewers in the UK" as below:

Whilst its mildly entertaining that ITV seem to think that Reading, Berkshire is not in the UK that is in fact the location of our customer. We did check the RIPE whois on the IP subnet used by our customer and it was indeed located in the UK so this is clearly just another ITV error. We simply advised that they contact This email address is being protected from spambots. You need JavaScript enabled to view it. as instructed because the only people who can fix this are ITV. 



Continue reading
  7693 Hits
  1 Comment
Recent comment in this post
Guest — LJM
ITV Player has always been a substandard piece of trash. Requiring logins, requiring ad-blockers to be removed when they aren't e... Read More
Friday, 13 January 2017 14:15
7693 Hits
1 Comment