SG Enterprise encryption - Bad Sectors survey

Not meaning to argue with the pros here - but maybe what MawfTech meant was this:

1. Create a file on the partition to be encrypted; make sure the file is contiguous (NTFS should do this for you; if not, defragment and repeat).

2. Set the file as immovable (possible on NTFS, and starting with Windows 7, also on FAT, AFAIK - see http://www.osronline.com/showthread.cfm?link=183894).

3. Write through to that file, and make a note you don't want to encrypt this one (should be easy, it is your driver working there).

4. Note the location of that file in sectors - regardless of the partition being encrypted or not, it is now sitting firmly on the hard disk.

5. Rejoice! You just got rid of an ugly hack involving bad sectors. No more dangerous chkdsk switches, no more extra sector math.

Risk? I say none special. If your user manages to mess with a file marked as hidden+system+immovable, to which only a special administrator account even has permissions, then good luck preventing him from doing pretty much anything else.

Precautions? Just one I am aware of - don't use PageDefrag. Should be easy to remember, at least as easy as the thing with chkdsk (which is installed by default, unlike PageDefrag).

So what do you say? Bad idea? Did I miss something? I believe GRUB2 uses similar technique (create a file and then access it as a sequence of sectors later on) if you tell it it doesn't deserve a partition - without any bad sectors involved. Not sure about the original GRUB, my brain got stuck on the concept of phase 1.5 - but my guess is no bad sectors there, either.

Petr

Petr,

You're spot on!

Actually had completely forgotten about Grub myself, good call. Was thinking more of the media player systems you get on e.g. Sony laptops or the IBM Recue and Recovery systems. Both create a small hidden partition (after main windows install) using exactly the method you describe and hide their systems inside that. There are many sneaky methods used for this that are signifcantly better than the bad-sector method (that's just copping out if you ask me). Now that Utimaco and Sophos have joined, why don't you guys start looking at e.g. the technology behind rootkits too.

Matt

Hi there,

seems like it might help to clear some things up in this thread.

MawfTech, you wrote:

Using that method, I now have no way of knowing impending surface failure on a drive without running a long diagnotic test.. By writing the kerneal area as bad sectors, you're now removing the ability for us guys to spot impending failures early…

Maybe there’’’’s a misunderstanding here. Opposed to your statement, we do not write anything as bad sectors, we only mark the protected areas as bad clusters in the file allocation table. So you got two different things here.

If you really want to check the physical health of the disks, there is no other way than to check for bad sectors (and if your drives feature S.M.A.R.T. technology, this will really ease your troubles, of course). Bad clusters are not a direct indicator for the physical health of a disk. This is also well documented for tools like chkdsk, and in the documentation for ScanDisk (http://support.microsoft.com/kb/q127055/) Microsoft even points you to encryption, among others, as an example of where you run into trouble when mixing up the two.

The remark of yours regarding SGE 4.x never used to do this may also be an indicator that the SGE bad clusters simply slipped by your health tests, which only adds to the point that you should try to directly test for the defects you’’’’re after.

petr_stipek, you wrote:

1. Create a file on the partition to be encrypted; make sure the file is contiguous (NTFS should do this for you; if not, defragment and repeat).
2. Set the file as immovable (possible on NTFS, and starting with Windows 7, also on FAT, AFAIK - see http://www.osronline.com/showthread.cfm?link=183894).
3. Write through to that file, and make a note you don't want to encrypt this one (should be easy, it is your driver working there).
4. Note the location of that file in sectors - regardless of the partition being encrypted or not, it is now sitting firmly on the hard disk.

The real challenge in software development, of course, is to find the simplest solution that reliably works.
1) is exactly what we are doing anyways.
2) to 4) may work, but it’’’’s brittle and complicated for a number of reasons, some obvious ones being that:
- our driver works on sector-level and, during normal operation, is actually not aware of any files at all, and
- what you describe is actually what bad cluster entries are designed for in the first place: keep your hands off these areas!

Why should we try to re-invent the wheel with a home-grown mechanism that the operating system is not even aware of? Doings so sounds like asking for trouble.

Some more food-for-thought, in addition to what I already mentioned:
- System-protecting files (as MawfTech suggested) works only on a running system, but not when you access the disk externally.
- Extra partitions may not only be cumbersome to create, especially if you need to downsize existing ones. They also may be deleted easily.

Hope that helps,

Oh...

_of_course_ we are talking clusters here, and not sectors - why didn't it hit me earlier?! [slapping my forehead] - Thank you for pointing that out, Thomas.

I would still maintain that using bad clusters to reserve a disk space is a hack, and not a particularly pretty one (MawfTech's assertion that it reduces the value of bad clusters as an early warning sign is valid, imho), but it sure looks like it became a kind of, er... industry standard hack? What do you know. Maybe it's just the naming and the fact that NTFS does not have an official standard - I guess if Microsoft called this feature "cluster exclusion" (clusterotomy, anyone? :smileytongue:) or "cluster blacklisting", instead of "bad cluster remapping", some of us would feel differently.

I am totally sure you guys do know what you are doing, and that this method is the lesser evil for some reason. At the same time, I am grateful this issue has been brought up so that now we all know there is a hack involved (industry standard, no less! :smileywink:) and what implications it has for system administration - quite manageable ones, I think, once you have this information. Maybe an explanation of what happens to your disk when you install SG should be included in the basic documentation, just to avoid confusion?

Petr

Hi Thomas,

It's good that you're engaging in this discussion, thanks.

Some followup points:

   - System-protecting files (as MawfTech suggested) works only on a running system, but not when you access the disk externally.

That doesn't really matter though does it. Accessing it externally is only done by your boot loader and kernel anyway, anything else doesn't need to know about it and can't because it's burried withing the encrypted partition table anyway. System-Protection prevents defrag and chkdsk doing anything with it and if it's contiguous then pagedefrag won't touch it either.

  -we do not write anything as bad sectors

But isn't that exactly what we see, 'bad sectors'.

Yes, SGE 4.5 does show a small area of bad sectors, 380KB which is much smaller than 98,000KB in SGN. 

Matt

Hi MawfTech,

you wrote about system-protecting:

That doesn't really matter though does it. Accessing it externally is only done by your boot loader and kernel anyway, anything else doesn't need to know about it and can't because it's burried withing the encrypted partition table anyway. System-Protection prevents defrag and chkdsk doing anything with it and if it's contiguous then pagedefrag won't touch it either.

Please don't forget to consider dual-boot systems. An OS might not protect against the deletion of such files located on another partition. Also, you could easily mess up the drive with a Windows PE CD with the SGN filter drivers. (Which would be your fault, of course, but I already hear voices which require protection against this, too.)

I want to point out another argument. The filter drivers would need to understand the file system sector level implementation if we do it the proposed way. In the end this would rock to the filter driver complexity and increase the likelyhood of bugs. I personally believe it is not a good idea to trade stability for a slightly better drive status detection (with having a S.M.A.R.T. alternative).

The bad clusters are our way to store pre-boot code. As with most software architectures, there are pro's and con's, and we seriously considered all the arguments in the design phase of SafeGuard Enterprise. Please believe me that all your arguments were discussed already years ago before we started coding.

With compliments,

Hi Thomas,

I hear clearly what you're saying but:

    >Also, you could easily mess up the drive with a Windows PE CD with the SGN filter drivers

Well you can do that anyway even with bad clusters so I don't follow that argument. I think if anyone started to mess at this level they really should be aware of the implications anyway. Even if we talked about non encrypted drives, if I mount a disk in another OS and deleted systems files, I'd expect the native OS to be screwed up. Equally, it might be important that the area needs to be manipulated to correct errors that occur from time to time in your software. Actually the greatest use I would have of the PE boot would simply be the removal of SGN rather than the manipulation of data. I much prefer to decrypt and then get windows to fix itself properly than try to tweak my own fixes, it's much more reliable that way anyway.

  >Please believe me that all your arguments were discussed already years ago before we started coding.

Technology moves on though Thomas and there are better ways now. You don't need to do anything differently with filers, at the moment you're marking a contiguos area of the disk marking bad and then storing the start-sector away in the boot sector (I'm assuming). Nothing changes there, you create a continuguous system protected area and again store away the start sector. It's exactly the same - no different filters anywhere just a different marker and one that doesn't imply the disk is going bad.

Pure curiosity, how does MS's Bit Locker (which I haven't tested myself yet) do disk encryption?

Matt

Hi All.  This thread is old but I thought I would add some more info.  3 Vista laptops went down in the last 2 days.  All got past the POA/PBA screen.  However they failed to boot into Windows.  Two gave specific file problems.  Examples: classpnp.sys, ksecdd.sys (both fairly common if you google them).  Apparently I set my policy up incorrectly (fixed now) - "User may only boot from hard disk" was set to yes.  This prevented booting from external media after POA - really bad.  I could not run the Windows cd to repair anything - ouch.  I was able to get Safeguard support to assist me in, "Slaving" the drives.  This means use a USB adapter with hard disk connections for your drive.  Excellent save on their part.  All 3 drives were readable.  I was able to recover the users' data.  Bad thing is we are rebuilding these machines again.  One of them will take the whole weekend to build (developer dude).  Despite my mistakes, I can't help think these issues are related to Safeguard in some way.  Maybe these laptops are just knocked around too much.  I will run checkdisk (as mentioned earlier in this thread) and report what I found in this thread.  Thanks.

Hi Jb1111,

Thanks for the post. Curious, did the WinPE boot system not get attempted. I'm a little surprised that wasn't the first choice for recovery rather than the slave drive. Am I missing something here, does the 'User may only boot from hard disk' actually prevent that from being able to open the drive?

Matt

Hi Matt,

the setting mentioned by JB1111 only checks if a user my boot from the CD after the POA/PBA. This will not restrict access to the drive in any way. However, wirh regards to this one I think that the plan of JB1111 was to boot after the POA to repair Windows somehow.

With regards to Recovery you can for sure also boot before the POA using WinPE and then access the drive via Recoverytoken as described in the manual. But even when doing so one will have to copy the data on a external hard drive so it might even be easier to slave the drive depending on the infrastructure that you have.

Regards

Dan