Tuesday, May 15, 2012

WhatsApp Forensics

Those who follow this blog may have noticed few months ago a post that introduced WhatsApp Xtract: this script was able to display in an HTML document all the WhatsApp messages extracted from an iPhone. And those who follow the xda developers forum may have recently noticed a thread on it.
This last month, thanks to Martina Weidner (aka ztedd) who has decided to take control of its development, we have obtained valuable results.

Intro:


WhatsApp is a widespread instant messaging application for smartphones, available for iOS, Android, BlackBerry, Symbian and Windows Phone. The chance to replace the traditional SMS service avoiding its cost, has allowed this application to gain popularity very quickly. The automatic synchronization of the app to the phone address book, the unlimited message length and the possibility to share an high range of multimedia attachments have persuaded many people... and who cares if it has suffered from some security issues!

Where to find the information:


As numerous apps do, WhatsApp stores all its information on a SQLite database: the location and the structure of the database are different from platform to platform.

Android

If you choose to avoid the rooting of the device (remember the digital forensics best practices!), you will be only able to get an encrypted file from the SD card (/sdcard/WhatsApp/Databases/msgstore.db.crypt). What's the file structure? Random, obviously! Is there a solution? Yes, there is... so far. The WhatsApp Database Encryption Project by Corjens, Spruyt and Wieringa has made known a vulnerability in the Android implementation of the AES cypher: the 192-bit key can be detected performing both static or active analysis on the software package. And the result is:

 346a23652a46392b4d73257c67317e352e3372482177652c

Just few Python code lines and a decrypted database can be obtained. For further information, read the project report.

Conversely, if you root the device, you will easily reach the plain databases (/data/data/com.whatsapp/databases/msgstore.db and wa.db).
As you can see in the first figure, the database of the Android version is splitted in two files: wa.db contains all the information related to the contacts (id, phone number, status, etc.), whereas msgstore.db stores the messages, including attachments.



iOS

iTunes has the capability to automatically synchronize and backup the iPhone content when you plug it with your computer. And the backup is not encrypted by default. So, also avoiding the use of the UFED physical analyzer, it could be possible to find on the computer of a person under suspicion, an enormous amount of data about his iPhone. iPhone Backup Extractor interpretes these data and enables you to extract all the files you want. Application/net.whatsapp.WhatsApp/Documents/ChatStorage.sqlite is the source of information we are looking for.



Unlike the Android version, the tables are here collected in a single file, but the structure is a little more complicated: ZWACHATSESSION and ZWASTATUS have the contacts, ZWAMESSAGE and ZWAMEDIAITEM collect the details on the messages and the attachments.


Main new features:


  • WhatsApp database can be inspected for both iOS (ChatStorage.sqlite) and Android (msgstore.db & wa.db) devices; 
  • Emoticons and attachments (images / video / audio / gps / contacts) are shown in the message content;
  • msgstore.db.crypt (Android) can be decrypted and inspected.

How to use:


  1. Download the archive and extract it to a certain folder on your computer, e.g. C:\WhatsApp;
  2. Copy the database(s) to e.g. C:\WhatsApp;
  3. You need Python and (for Android msgstore.db.crypt decryption) the PyCrypto library;
  4. Run the .bat files provided with the package or type in the console one of the following commands:

    For Android DB:
    > python whatsapp_xtract.py msgstore.db -w wa.db
    OR (if wa.db is unavailable)
    > python whatsapp_xtract.py msgstore.db
    OR (for crypted db)
    > python whatsapp_xtract.py msgstore.db.crypt

    For iPhone DB: (-w option is ignored)
    > python whatsapp_xtract.py ChatStorage.sqlite

To do:


Currently, iOS database analysis could be improved in the handling of attachments (gps / audio) and group messages. We would also extend the support to Nokia and Blackberry versions of WhatsApp, but we don't have enough information on them. Anyone wishing to contribute, with ideas and ... databases to analyze, is welcome!

Follow the updates on the Hotoloti repository and on the XDA Developers Forum!

57 comments:

  1. What about BlackBerry? Is it possible to get the Whatsapp messages from a BB and read them?
    Thanks! this is a really good post!

    Carlos

    ReplyDelete
    Replies
    1. Hi Carlos.
      AFAIK there isn't a way to get an unencrypted messagestore.db file from Blackberry Whatsapp!
      For this reason, so far it is not supported..
      Thanks!

      Delete
  2. Thank you Carlos.
    We'll expand the work on BB too as soon as possible, even if the extraction ("acquisition") of WhatsApp chat storage and its analysis are two different issues. Fabio, who is the tool creator, could provide to you more details: feel free to write directly to him.

    ReplyDelete
  3. I was wondering if the timestamp is correct? Whats the output of the Timestamp? is it UTC time? or GMT?

    ReplyDelete
    Replies
    1. Hi palojoe!
      In the Android version, Whatsapp stores the timestamp in Epoch format, whereas the iPhone uses the MAC absolute time (for this reason you can note the displacement value '11323*60*1440' in the code: it is the difference between the epoch and the mac absolute starting dates).
      The function datetime.fromtimestamp() gets the epoch timestamp and returns the local time, according to the current time zone environment setting.

      Delete
    2. Hi Fabio,

      How do we convert Whatsapp message iPhone backup time stamp 34323-11-16 16:10:29.106 and 34323-10-11 12:00:00.000 to real time? Is therany method or formula.

      Delete
  4. I would to say thank you for this amazing tool!!

    ReplyDelete
  5. Hi id just like to say please please please is there anyway you can create a tool that will let us view blackberry whatsapp database

    thanks

    ReplyDelete
    Replies
    1. I'm still unable to inspect the BB whatsapp database for two main reasons: 1 - I don't have a BB and 2 - The BB strong encryption probably prevents the messages extraction. As soon as I've news, I'll let you know!

      Delete
  6. yu wa, did you manage to get info the 'blackberry whatsapp database ?

    ReplyDelete
    Replies
    1. To extract messages from a BB WhatsApp file you must have it in clear text. WhatsApp Forensics is not able to decrypt an encrypted Whatspp file (coming from a BB, at least) but it will be able to parse it once in clear text.

      Delete
  7. Hi, may I know if this tool recovers delted whatsapp messages ?

    Thanks,

    ReplyDelete
    Replies
    1. Hi Ben,
      the tool extracts all the messages stored in the whatsapp sqlite db. The deleted messages are unrecoverable, unless you have older backups of that file. If you have an Android device, check the folder /WhatsApp/Databases into the SD card.

      Delete
    2. Thank you fabio, may I know what does the values for 'Msg Status' and 'Media Size' means in the generated html report?

      Delete
    3. 'Media size' is the size (in bytes) of the message attachment (image, video, etc.).
      'Msg status' gives information about the message transfer. I didn't find any documentation about the meaning of its values, but according to my observations:
      0 = Read locally
      4 = Unread by recipient
      5 = Read by recipient
      6 = Group-related info (e.g. new group name/image/partecipant, etc.)

      Delete
  8. nah did not get a reply... whatsapp db are tricky as they are encrypted...

    ReplyDelete
  9. Has anyone anywhere figured a way to decrypt the blasted blackberry messagestore.db yet!? This seems an impossible task and I refuse to believe that this is indeed the case! Any help please!

    ReplyDelete
    Replies
    1. Unfortunately not. See reference here https://www.os3.nl/_media/2011-2012/students/ssn_project_report.pdf

      A quick note: even if you disable cryptography (on the SD card and/or at all) the database will be encrypted.

      Delete
  10. Hello,

    How about Symbian whatsapp database?

    ReplyDelete
    Replies
    1. Hi,
      you can try to hack your symbian phone and get the whatsapp application data from the 'Private' system folder of your device. They should be unencrypted. Unfortunately WhatsApp Xtract still doesn't support symbian devices, because data are stored in cdb files (constant databases) instead of sqlite3.

      Delete
    2. Hello,

      I need urgent read to nokia whatsapp chat logs.But I dont know.all sql editor programs dont read it. Can I read cdb editor ?

      Delete
    3. We didn't worked on nokia cdb. Actually WhatAppXtract works only on sqlite3 files.

      Delete
    4. is it possible nokia cdb files reading ?

      Delete
    5. WhatAppXtract actually works on iOS and Android sqlite3 db: no other OSes are supported.

      Regarding your question: I'd take a look at Noki.

      Delete
    6. I contact with Noki. But he cannot read it. Do you have a plan read Nokia database ?

      Delete
    7. You can also look at NBU Explorer (http://sourceforge.net/projects/nbuexplorer/)

      Delete
    8. And no, actually we have not any plan to read Nokia db.

      Delete
  11. How about nokia serie c?

    ReplyDelete
  12. Nokia whatsapp chat logs can be read in notepad. That is the only programme I found, it contains XMPP records 4 what happens throughout the day. No message info, but you can kinda make out what happens by reading it and looking for patterns. Shows numbers. Who sent who a message. You can see when whatsapp was activated/deactived and what was view ie. Favourites, chat from some1, etc. Anyone know of anything else?

    This is the closest I got to seeing what is happening on whatsapp on nokia, maybe someone else can comment on a better way?

    There is way to little info to gather on whatsapp nokia :'(

    ReplyDelete
    Replies
    1. Thanks for your insight Shanaaz! Unfortunately we are not working on Nokia, neither we plan to do. But if you find a "solution", please let us know: we could add it to the software. Kind regards.

      Delete
    2. What's the programm name ? Could you share us Shanazz?

      Delete
  13. hello.. im having some problems
    first. i cannot see emoticons, but pictures send yes
    second. i used the drag and drop, i was thinking if the emoticon doesn't appear there.
    third. in the crypteb.bat says at the end: could not open database file.. but as i said before, i did it in the drag and drop.. it's the same result?

    note: i just copy the ''database'' folder
    thank youu

    ReplyDelete
    Replies
    1. Don't know exaclty what do you mean with 'drag and drop'.
      Please follow the 'How to use' instructions.

      Delete
    2. Hi! If you have and android device and the need to decrypt the db, you can simply put the msgstore.db.crypt in the main folder of Xtract and run the "whatsapp_xtract_android_crypted.bat". Alternatively you can open the cmd prompt and type "python whatsapp_xtract.py msgstore.db.crypt".
      Be sure to have the latest release of Xtract.. anyway, it's possible that our set of emojis is not updated with the most recent ones.

      Delete
    3. Any luck yet on db file reading for blackberry??

      Delete
  14. sorry, im using the program whatsapp extract, i saw your name in there.. so i though you could help me.. or am i in the wrong place

    ReplyDelete
  15. ok :) thanks.. the thing is.. i can't see emoticons :/.. how's that?

    ReplyDelete
    Replies
    1. As Fabio wrote, "it's possible that our set of emojis is not updated with the most recent ones".

      Delete
  16. Ive tried all possible angles at the blackberry .bak files and database files, closest ive come sofar is that i was able to get it in Chinese or Korean. Any advancement regarding the blackberry.

    ReplyDelete
    Replies
    1. I couldn't understand. No BB advancement.

      Delete
  17. I've not fully understood: have you been able to decipher them?

    ReplyDelete
  18. Hi,
    Do you hv a video on this. Im not an IT expert, thus confuse on ur explanation. Really appreciate ur assitance. Im using android.

    ReplyDelete
  19. I know it has been asked before, but this question is from a slightly different angle, so I hope you can help ;-).

    If I have the password that the database was encrypted with in my blackberry, is there a chance to decrypt it?

    ReplyDelete
    Replies
    1. Hi Thomas,
      never tried but seems interesting. Have you tried with BB simulator?

      Delete
  20. "Msg status" reverse engineered from android version :

    0 = "STATUS_UNSENT"
    1 = "STATUS_UPLOADING"
    2 = "STATUS_UPLOADED"
    3 = "STATUS_SENT_BY_CLIENT"
    4 = "STATUS_RECEIVED_BY_SERVER"
    5 = "STATUS_RECEIVED_BY_TARGET"
    6 = "STATUS_NEVER_SEND"

    ReplyDelete
  21. Your python script works under Linux too I presume?

    Will definitely test it out if it works under Linux :)

    ReplyDelete
  22. Hi !
    Firstly, thank you for the great info you gave regarding the decryption key and the tool to show the database in a readable format.

    I have a question ; do all androids use the same decryption key ? i.e
    346a23652a46392b4d73257c67317e352e3372482177652c

    Its important for me to know this because I intend to quote this in my work if that is true. I read the "https://www.os3.nl/_media/2011-2012/students/ssn_project_report.pdf" paper too for this. They do not have their contact info like email id mentioned in the paper that I can contact them.

    I was wondering if you have any more information related to the decryption key for whatsapp db on Android and can confirm about the same key being used for all android phones.

    Your reply would be highly appreciated.
    Thanks,
    Neha

    ReplyDelete
    Replies
    1. Hello,
      I tested the key, on the latest version of WA on android, and it still can decrypt the database.

      However, I tried to reproduce their steps but I can't understand their process after page 13.
      I found the class 'a6' that they mentioned but in my case the name is different. Parameter names are also different.
      It's really difficult for me to understand their process and results after figure 4.7 on page 13.

      Delete
  23. i have my symbian whatsapp message store folder..but i have upgraded to an android phone cos other one broke..i really need to be able to view my symbian whatsapp messages and cannot find anyway to do this...can anyone help me..very important
    please

    ReplyDelete
  24. Hello,
    I have a question about the thumbnails.
    Where are they stored in the database and how do I view them without generating the html report?

    Any reply will be appreciated greatly.thx!

    ReplyDelete
  25. you may locate them on blackberry\system\media, and try bitmaprip.exe (a freeware app). Install it on a new folder with the thumbs file, open cmd and on command line type bitmaprip + file name. Luck!

    ReplyDelete
  26. hi, first thanks on this great tool, I used it a month or so ago it worked fine, but now, I try to extract my ChatStorage.sqlite file (58,432KB) the extraction starts normally but it stops suddenly with the html file only 26,534KB.
    the cmd shows the following:

    printing output to ChatStorage.sqlite.html
    c:\documents and settings\user\desktop\folder>pause

    I tired it on a different device faced the same problem, can you offer some help please?
    thanks in advance.

    ReplyDelete