I recently needed to recover passwords from a Linux system where I had the drive which I could connect to a Windows PC but this presented several issues starting with finding the right file then what tools to use and most importantly how to mate it correctly in OpenCL mode to get the benefit of graphics card processing power!
Firstly the drive was formatted as EXT3 which Windows doesn’t natively support. After a bit of research I found a free program called Ext Volume Manager and gave it a go. It worked perfectly and after giving a list of available drives you can double click and mount the drive as a drive letter in Windows then just browse to it like any other drive. It was simple and worked really well.
Now that problem out the way we needed to find the password file. In Linux passwords were historically stored in a hashed form in root/etc/ in a file named passwd so this is the first place to look. Open it in notepad or similar and it is highly likely you will see a series of lines line this:
The X is where the hash would have been found historically but when the security was updated this method was changed and so the X just shows that there is a password configured but it’s stored elsewhere.
That elsewhere is a file in the same location called ‘shadow’. The structure of this file is very similar to ‘passwd’ but in Linux has different permissions. Luckily in windows this doesn’t make much difference so we can just open it.
The data should look more like this (I have cropped out some of the line to avoid it filling the screen. The $6$ in this case identifies the password hash as being sha512crypt format but yours may differ, the options are:
$1 = MD5 hashing algorithm.
$2 =Blowfish Algorithm is in use.
$5 =SHA-256 Algorithm
$6 =SHA-512 Algorithm
The next bit ‘ THMmaDC5 ‘ is the ‘salt’ value which is random data used to encode the password as the hash making it more difficult to guess.
The remainder up to the colon is the hashed password which is what needs to be guessed so now we have the right file.
Next go ahead and download Cygwin ( https://www.cygwin.com/ ) this is basically a miniature Linux platform on Windows which lets you compile Linux programs to run under Windows if they are compiled for it.
When installing Cygwin generally you can just use defaults and whatever local mirror you fancy however when the list of tools is shown search for OpenCL and add this to the installation.
Add the highlighted component to the install and continue and you should find you will soon have a Linux installation in a folder on your PC (default location is C:\cygwin64\ ).
Next download the zip of latest version of John the Ripper ( https://www.openwall.com/john/ ) – this is a widely recognised tool for this purpose and seems to work best. I also tried a program called HashCat but this didn’t seem to be able to find the hashes in the file. The version I used was 1.9.0-jumbo-1-64 Bit.
Hopefully the zip will look like this. Copy the all the folders and paste them into the Cygwin folder – there will already be a folders with those names so merge them. This operation adds the descriptors to allow Cygwin to recognise your OpenCL device however on my PC (and from what I’ve been reading online several others the path was incorrect so we’ll fix that.
Browse to the following path C:\cygwin64\etc\OpenCL\vendors\ and open the amd.icd file in notepad.
Next go the the system32 folder as shown and search for ‘amdocl64.dll’ . In my case this wasn’t present in the system32 folder directly but I found a match in System32\DriverStore\FileRepository\ . If that is what you find just copy the file and paste it into system32 itself and this should correct the mismatch.
Next copy your ‘shadow’ file into ‘C:\cygwin64\run’ – technically this isn’t required but it makes life easier. In my case I edited it to have a .txt extension to make testing easier. Now to test it!
Open a command prompt Window and browse to ‘C:\cygwin64\run’ then enter the following command:
john shadow.txt –format=sha512crypt-opencl
Interchanging the format for whatever is relevant to your hash type. If you run john without specifying a hash format it will recognise it correctly but will default to CPU only mode rather than the OpenCL version which comes with a performance hit for most people.
All going well you should see something like this :
That tells us its working fine and has successfully found the graphics card as a processing device. Now take a break and leave it to churn through its options for as long as it takes. It won’t be fast!
Apologies to anyone who’s been waiting for an update on any of the projects I’ve put on here – I’ve had a lot going on over the last year and writing up all of this information takes me a lot of time so has taken a bit of a back seat. None of that information has been lost and I hope to start catching up on all of this over the next few months.
Where we are right now :
V6 Mazda RX8
The V6 engine has been fully rebuilt and has been sat in my living room next to an engine crane for about 8 months now, Partly this is due to me not currently having a garage to work in and it being winter and partly due to being very busy last summer. But yes, the project is still ongoing and hopefully will make good progress this year.
Six million dollar welder
Is still on my workbench awaiting the last couple of bits. I’ve had to swap out 24V PSU’s a couple times as they couldn’t deal with the current of the new feed motor. It now works but needs the relays mounted to the panel so I need to remember to find 4″ of DIN rail.
Is still working beautifully after the Arduino fan controller mod. It’s been hosting this blog ever since so something like 12 months now without a problem. It also provides the storage for my CCTV system.
I’ve got a couple of updates to write about expanding the system beyond the original one camera setup I wrote up last year. Using multiple cameras with network storage seems to be poorly documented and have a few interesting quirks but after some work I got it working fine. The biggest problem I’ve had since is when the local police asked for my footage following a nearby break-in I had to supply them on a 1 Tb hard drive! High resolution digital video takes up a lot of space!
I’ve had various other projects going on as well. I’ve started looking into using NodeMCU WiFi microcontrollers as sensor nodes with their own web servers around my house with NodeRed requesting the page and parsing the HTML to return the information I want. I can read the temperature sensor connected to the nodes and control the LED on the MCU from NodeRed. Currently the data is not stored because I ran out of time trying to get MySQL setup. I’ll write this all up at some point
Raspberry Pi’s – I seem to have collected a selection of Raspberry Pi’s somewhere along the way. I have a Zero, a ZeroW, 1B (which in a different life I turned into a PoE CCTV camera), 2B and 3B plus I think somewhere there’s another 2B. I really should do something cool with them!
Wooden storage box – I started restoring a large wooden box some time ago. The box used to be in my granddad’s workshop as a toolbox for many years and before that I gather it belonged to his uncle so it’s been around for a long time and has suffered a bit with use and age with some areas with woodworm damage and the rope handles badly degraded. It has also been painted brown at some stage so I’ll need to get that cleaned up as well. I intend to restore it to a ‘usable’ condition so rather than trying to make it as new just tidy it up, repair the damage and make it solid enough not to degrade further but still look like the well used item it is.
Subwoofer project – Has been in constant use for ages despite never being technically finished. I really need to actually finish it and write this up!
There will be more but I think that’s enough update for now – rest assured I’ve not stopped! Especially since this morning another turbo arrived in the post, the fourth I now have here….
Recently one of my neighbours told me there had been an incident where someone was seen trying van doors along the street late at night and combined with the habit of numerous people attempting to turn their cars round on my driveway with seemingly no ability to successfully do so or any apparent awareness of their surroundings I decided it was probably a good idea to get security cameras to keep an eye on the place!
I started looking into it and decided that network type cameras were the best idea because I already have a suitable server to store the footage and a long time ago started running substantial amounts of CAT5e cabling all over my house because due to having thick stone walls WiFi from a normal home router really wasn’t an option. Since I needed to install three separate wireless access points to cover all the indoor areas and so needed to run cable across the house anyway. With (at least some!) of the cable run in it sounded like the best solution.
At this point I should probably point out that I know nothing about what is the best camera, I can’t say for sure if what I bought were actually the best option but they seem pretty good so far but your mileage may vary.
So after a bit of searching I found that Hikvision cameras get good reviews and are generally quite well regarded so when a couple came up on eBay for a decent price I went for it. The model I bought was the DS-2CD2055FWD-I.
Hikvision DS-2CD2055FWD-I 5MP Camera
This is a 5MP camera which supports power over ethernet according to the 802.3AF standard but also come with a connector to allow the unit to be powered from a standard 12V wall adapter with a barrel plug. These cameras have an on board micro-SD card slot so can record both internally and to a network drive/DVR.
Because I didn’t have proper POE set up I decided to use a passive POE injector/splitter set from eBay, these are available for a couple of quid and allow power from a barrel connector power adapter to be merged with 100Mbit data onto a standard CAT5e cable. They have limitations but for low power over short distances they’re quite useful and let me keep the power supply safely inside.
These aren’t weatherproof so I suggest if you’re using them tuck them somewhere sheltered or waterproof them. Or just buy a switch with proper POE support!
Ok so once you having network and power to the camera next you need to start configuring it. From a computer attached to the same network you need to install the Hikvision utility SADP which can be downloaded from here:
This will scan your network for active cameras and display them in a list. just click the check box on the camera you want to configure and enter a password – make a note of this password – you will need it to make further changes to the camera later on. Then you can set up the IP parameters for the camera. More details on this process can be found here:
The rest of the camera configuration is generally as per how you want it for your specific use, the interesting bit comes when you try to get it to connect to a network drive as there seem to be a few issues.
Hikvision cameras don’t seem to have a good implementation of network drive access, the main problem being that if presented with a network drive of over a certain size (no-one seems to know what size but it is in the order of 100’s of gigabytes rather than terabytes) the camera fails the initialise the network drive. Creating a network folder with a quota doesn’t work either, windows devices will show the quota limit as the drive space but the Hikvision camera will still show the total volume size. We need to limit it in a more low level way!
The best solution I found under windows was to create a virtual hard disk in the machine you want to be the server. This is done by opening the windows disk management tool and selecting action -> create VHD then you just choose the appropriate size and where you want to save the virtual drive file. If you choose to create a fixed size a file of the full drive size will be created at the chosen location. I created a fixed size VHD of 200 Gb on my storage drive and this seemed to work well but a dynamic would probably work fine. The new drive will be shown in the drive list and should be initialised, right click on the new drive and initialise it as MBR. Do the same again but this time create a new simple volume of the full drive size and format it.
Once the new drive is created it will be shown in windows explorer and look just like any other drive. Next you need to share the drive over the network which in my case involved creating a second account on the server (or whatever PC you’re storing the data to) with limited access to do anything to the PC. Then the new drive is shared under this user account on the network. The username and password are needed to allow the camera to connect so make a note of these.
Next navigate a computer on the network to the IP address you configured for the camera and when prompted log in. Go to the configuration tab and under “advance configuration” select storage. and go to the “NAS” or possible “NET HDD” tab depending on version. Enter the IP address of the server and under file path enter the the network folder you shared earlier but prefix it with a forward slash. For example if you shared the drive under the network name Video then enter “/Video” without the quotes in the box. Mount type will usually be “SMB/CIFS” then username and password as you configured earlier for the user account on the drive.
Not shown in this picture is the “test” button next to the password, click test and hopefully you will get a message telling you testing succeeded.
Next you go across to the “Storage Management” (or possibly “HDD Management”) tab and with any luck the drive you just added will be listed. Click the check box and click format.
Once that completes it now has a network drive to write to!
Other points to note:
The camera will keep as much footage as it can in the space it has but will remove older footage once the space is full. You can have multiple destination drives but the camera will write the most recent data too all of them with the smaller spaces retaining less far back so you can have a camera with say a 16Gb SD card internally and a 200Gb network
5MP footage takes up a lot of storage space so I recommend creating a separate virtual drive for each camera if you spare the space!
You might notice if you use Chrome that you cannot get a preview from the camera – this is due to the camera using old plugins that are no longer compatible but if you search for a Chrome extension called IE Tab and install it adds a new button next to the address bar. Click the new button and go to the camera login again and you should find you get a preview!
If you just want the CANbus ID for the RX8 PS light and not all the background work just skip to the end!
So it’s been quite a long time since I had chance to have another go at getting the CANbus on the cluster working and while previously I manged to get everything apart from the power steering warning light working I decided I really should find out why not. This is a simple lesson in why getting sufficient sleep is really important!
I was contacted a while ago buy a guy doing a similar conversion to mine who happened to have a second complete RX8 and a variety of diagnostic equipment that can talk CANbus who had send me a block of CANbus log data that I’ve done nothing like enough work on since I’ve had it! Anyway the key here is I knew that the car some of the logs had come from had the factory PCM (ECU) working as intended and as such the power steering light doesn’t illuminate. This meant that somewhere in that log file was the data that turned off the light, I just had to work out where it was!
Now first off I took the approach of taking the two sets of data logs I had, one from a car with a functioning PCM and one from a car that doesn’t. Then list out all the ID’s that occurred in each set of data. I’m going to assume for my purposes that any that occur in both sets are not required. The logic being that the data that turns off the light must be present in one set and not the other. I admit that this might not be the case if there’s something more complex going on like if with the PCM removed the power steering computer doesn’t get the required data and sends a different value on the same ID. But for now it’s a starting point!
The ID’s that remain are as follows:
201, 203, 215, 231, 240, 250, 420, 620, 630, 650
A couple of these we’ve already seen elsewhere, specifically 201, which is the RPM and speed to the dash, and 420 which control the cluster warning lights. So after setting up the arduino and canbus data to strobe all these remaining address and nothing happening I gave up!
Many weeks went by and I it was nagging at me why I couldn’t find the right code. Eventually I decided to try a different tack so I ordered a USBtin which is a really neat little USB to CAN adapter which appears as a virtual COM device in windows and can be controlled using a series of commands known as the LAWCIEL protocol (details of which can be found here). The kit version is really quite cheap and would probably be a good option for the budget conscious but on this occasion I just decided to be lazy and buy the prebuilt one.
Clearly I’ve decided to PC control it at this point! Next up I needed a way to stop it when the warning light went off. I ordered a very cheap optical detector off ebay which can be wired into an Arduino or something similar. These are the ones that vary around £1-2 so difficult to argue with. They offer a potentiometer to adjust the switching brightness so I can tune it to what I need and a digital output so I don’t need to mess about doing analog reads or calibrating things on a microcontroller. Yes i know it’s not the neatest or most efficient way of doing it but for my purposes it’s so cheap and easy it really doesn’t matter! So I need to make that talk to a PC in a way I can use and that’s where this whole thing starts to get more interesting.
I’d pondered ways of interfacing a microcontroller to a PC easily and while it’s not terribly hard to make it shout over serial when an input goes high I came across something much more interesting. There’s a company called National Instruments who make lots of very expensive software and equipment for recording data from experiments but fairly recently they started supporting hobbyists by producing the Labview Makerhub, and more specifically a package called LINX. LINX includes a series of drivers and firmwares to allow things like Arduinos, Beaglebones and even Raspberry Pi’s to be used as IO devices (the Raspberry Pi can actually have programs loaded to them as a full remote node). This is quite a major step because it suddenly allows hobbyists to use really good professional software without having the problem of only being able to use NI’s extremely expensive hardware! This gave me and idea – use labview as the core software then I can use the supplied LINX firmware to set up an arduino as IO. To make this deal even sweeter you can also download Labview for free from NI for home use. Take a look here
So after a quick bit of following the instructions we have a basic labview program that will read the arduino IO via serial:
Basically what this does is it starts by opening a serial connection via the LINX toolkit, this returns a device name to an on screen string display and passes a reference which identifies the connection to the read stage. The next bit the larger grey rectangle is how Labview handles a while loop so it’ll keep performing the enclosed functions constantly from left to right until the conditional term in the bottom right goes high – in this case it’s a stop button. So basically the loop just calls a LINX channel read of channel 2 where I connected the light sensor to the Arduino. The inner rectangle only executes when the read value is false (i.e. when the light goes off) and while there’s a lot of information recorded here from elsewhere in the program basically if it sees the light go off it records the current ID being tested, the time that has elapsed since the test started. This means we know when it’s right!
Labview is designed to capture data from lab instruments and so there’s a really handy thing called the VISA toolkit that allows blocks of data read and write via the serial port and basically you can just open a port with specified settings then make read and write requests and do things like crop the incoming data at a predefined character. In this case that character needed to be CR (Carriage Return) this is ASCIIcharacter 13 because LAWCIEL terminates everything with one.
For the USBtin we open the correct COM port at 115200,8 Data bits, No parity, 1 stop bit and no flow control. The other thing to note is at the top right, this sets the termination character to the numeric value of CR, the benefit here is you can perform a read of any length and it will automatically break up the data in the buffer so a single read can vary in length but the start will always synchronise with the read call. Opening the connection in a terminal program for the first time and you’ll see nothing actually happens as such, an OK is signified by a CR so all you see is the cursor move. At this stage we are only connected to the USBtin, not the CANbus. So next, configure the CAN connection, send a value of “S6\r” . The code is S6, this will set the USBtin to 500kbit correct for the dash, the \r is how you indicate a CR in a Labview string. Next I chose to send “v\r” which requests the version from the USBtin, we don’t need this but it gives a solid confirmation it’s talking to us. Next up Z1\r tells the USBtin to timestamp the incoming data, I thought this might be useful but never actually implemented it.
With the setup complete we can start reading data by opening the CAN connection by sending O\r. On a terminal program (assuming you have the CANbus wired up) doing this would result in packets of can data from the cluster appearing. The initial read of length 1 byte reads just the confirmation from the USBtin that it has received the open request. Next is the main read, it’s worth noting the read length is set at 50 bytes but this will be cut short by the termination character set earlier so we can accept varying length CAN data. C\r closes the CAN connection and again another read 1 byte clears the acknowledge. Tacked on the end is a section to read the controller status looking for error states etc. The keen eyed amongst you will notice the majority of this code is conditional, this is because the code needs to insert send requests among the stream of reads. This is because if the data is not read from the USBtin constantly a buffer somewhere fills (I imagine on the USBtin itself but can’t confirm this) and the port crashes. I spent a lot of time finding this out the hard way!
This is the write data code, again very similar but it just opens the port, writes whatever string is in the buffer and closes the connection. Once the connection is confirmed closed it resets the Boolean that causes the ‘write’ condition so on the next loop it goes back to reading again to keep the buffer clear. The read loop runs at the maximum possible speed but it is slowed down because it waits for either a termination character or 50 characters to be received before it completes and loops again.
Beyond that the only other bits of code just generate the data for the write buffer using an increment counter for the ID field and toggling between either 8 bytes of FFFF or 0000 every 100ms for 20 cycles and setting the write flag high to send the data..
So after letting this run for a fair while it started spitting out values, specifically the ID 300 for the power steering light. Wait a minute that wasn’t in the list earlier. Yes I know that, that’s where getting enough sleep comes in. Originally I split the data based on whether or not the PCM was fitted and ignored the ones that occurred in both sets, the obvious mistake here is that of course the power steering light isn’t controlled by the PCM, quite logically it’s controlled by the power steering controller!
So there we go, ID 300, the first byte (leading) controls the light, values below 80 turn the light off. Unplugging the PCM causes the controller to send 80 on loop hence the the warning light.
Get data from ID: 4B1
0 0 0 0 0 0 0 0
Get data from ID: 81
43 6F 1 4B
Get data from ID: 300
Get data from ID: 231
F 0 FF FF 0
Get data from ID: 430
95 5F 83 0 0 0 60
Get data from ID: 81
25 6F 1 4B
Get data from ID: 81
16 6F 1 4B
Get data from ID: 630
8 0 0 0 0 0 6A 6A
Looking at the log data again we see that ID 300 is getting a value of 80 – this is during the self test before the engine is started. I previously tried sending this data on the original Arduino CAN setup and go no result so what did I do wrong. Again this is based on another assumption, I though the logger was ignoring high order bytes with zero value (ie this value if could be 00-00-00-00-00-00-00-80) well it turns out that was totally wrong, it actually ignores trailing zeros, the value of the can data here should be 80-00-00-00-00-00-00-00.
So while all these hours of work told me one thing I should have already known it’s actually worked out Ok because it highlighted this other problem (read ‘incorrect assumption’ !). This means The odds of me working out all the other data from the logs (that I’d previously written off as not usable) is actually much higher!
Just a quick post to record a solution I just found to a problem I’ve had for ages.
The problem revolves around the AMD crimson drivers when using eyefinity. What happens is the start bar will default to being across all 3 screens on eyefinity setups on boot even when the driver options are set up to force it to the one screen only – usually the centre one. Opening the graphics advanced settings will set it right until the system is rebooted but this is an irritating thing to have to do on every reboot.
Right click in the window and click new—–>shortcut.
As the location of the item paste in :
For 64 bit
“C:\Program Files (x86)\AMD\CNext\CCCSlim\SLSTaskbar64.exe” -l -e
For 32 bit “C:\Program Files (x86)\AMD\CNext\CCCSlim\SLSTaskbar.exe” -l -e
Then just click next and finish. If you run the shortcut now you should find your start bar jumps to the correct monitor and will do the same every time you reboot so no need to manually fix it any more!
Hopefully this will help someone else with the same problem!