Restoring SMS Backup from CSV to Blackberry devices
The tool in this article, CSV2IPD, reads text messages from CSV files and generates an IPD file that can be imported to Blackberry devices by Blackberry Desktop Software for devices running Blackberry OS 7 and older, or Blackberry Link for devices running Blackberry OS 10.
UPDATE (8 October 2014): I received a few comments from several readers stating that, although Blackberry Link recognized the IPD file as a legacy backup file and seemed to restore the messages successfully, no messages actually appeared on the device.
At first I suspected issues with the IPD file format generated by the CSV2IPD tool or some settings on the devices which delete old message, which turned out not to be the case.
After further research I found a hint here which pointed to a KB article by RIM describing the issue, KB34963 (SMS text messages from a BlackBerry smartphone running BlackBerry Device Software 4.5 to 5.0 or BlackBerry 6 to 7.1 are not migrated to the BlackBerry 10 smartphone). Although the English KB article has since been removed, I managed to locate a French version of the article, cached by Google. After reading the French KB, it seems that the problem is due to a Blackberry Link bug which results in all text messages not restored if an area code is not set on the OS 10 device. The solution to this is to set an area code, even if you do not use one, using the following steps:
1. On the Home screen, tap the Phone icon.
2. Slide your finger from top to bottom of the screen.
3. Select Settings > Smart Dialing.
4. Enter a three-digit area code (e.g. 214).
5. Select the Back button to save the changes.
With the area code set, the restore of the text messages should be successful and all messages in the IPD file should appear on the device. If you have a lot of messages, it may take a few minutes for the text messages application to update the message list – so be patient.
If the area code does not appear to be saved, select a country (e.g. United States) that is known to have different area codes for different cities. Blackberry OS 10 devices keep a list of countries that do not have area codes (e.g. Singapore) and will not save the area code settings if a country in the list is selected. After the SMS restore is completed, you can always set back the correct values.
With this workaround, all text messages from the IPD file generated by the CSV2IPD tool (and from the official Blackberry Desktop Software application) will be restored successfully on to the device. The format of the generated IPD file remains unchanged and compatible with Blackberry Desktop Software and other IPD utilities.
Take note that this issue only affects text messages restored from an IPD backup file while messages from a BBB backup file (newer format supported by Blackberry Link) are not affected. Hence the above workaround will not be necessary to restore messages from a backup file created by Blackberry Link, which is in the new BBB format, or for users using Blackberry OS 7 or older.
Downloads
CSV2IPD
Requires .NET Framework 3.5, which will be automatically installed on Windows 7 and above. If you are using Windows XP, you can download it here.
Change history
- 8 Feb 2015: Fix a bug where CSV2IPD would sometimes fail to generate the IPD file due to folder location issues. Fix some library issues and CSV2IPD now requires .NET framework 3.5 to run properly. Also apply some minor UI changes with better wording to make it easier to understand.
- 24 Mar 2012: Add text fields to allow users to specify a custom date format to be used for conversion. The format must be useable by .NET’s DateTime.ParseExact function, refer to this for details.
- 20 Jan 2012: Better support for Unicode characters. Sample CSV files with Unicode messages in various languages are included in the download. CSV2IPD will also report the number of successfully imported messages at the end of the conversion, not just the number of invalid records.
- 18 Nov 2011: Use .NET DateTime class instead VB6 CDate function to parse date strings (which may fail on non-English systems). Also added a log file (CSV2IPD.log) that can be used for debugging.
- 7 May 2011: Following a request from a reader, the program has been updated to handle CSV more robustly, supporting multi-line messages and messages having semicolons. It also reports the number of invalid records in the CSV file after the conversions. The source code and binary files have been updated.
- 1 Oct 2010: First release
Background
Following my previous post on a Windows Mobile tool called SMS2IPD that would run on a Windows Mobile device and creates and IPD backup of all the SMS that can be restored to a Blackberry device via Blackberry Desktop Manager, a German reader contacted me asking how he could do the same for his Nokia phone.
Being no Symbian expert who could write a similiar tool running on Symbian phone, I advised him to use Nokia PC Suite to create a CSV backup of all his SMS. After that, I have written a tool that will generate an IPD backup from the CSV files.
The tool, CSV2IPD
The program, called CSV2IPD, accepts 2 CSV input files for Incoming SMS and Outgoing SMS, as demonstrated in the following screenshot:
The CSV for Inbox looks like:
sms;deliver;"+987654321";"";"";"2010.08.25 15:14";"";"Message 1"
sms;deliver;"+123456789";"";"";"2010.08.25 15:15";"";"Message 2"
sms;deliver;"+111111111";"";"";"2010.08.25 15:17";"";"Message 3"
The CSV for Sent Items looks like:
sms;submit;"";"+0123456789";"";"2010.08.25 15:15";"";"Hello world 1."
sms;submit;"";"+0123456789";"";"2010.08.25 15:16";"";"Hello world 2."
sms;submit;"";"+0123456789";"";"2010.08.25 15:17";"";"Hello world 3."
If your exported CSV file via Nokia PC Suite does not look exactly as above, you can use a text editor tool such as Notepad++ then use Find/Replace, perhaps with the help of regular expressions, to modify it. Make sure you do not have extra quotation marks at the beginning and the end of each line. If the delimiter is comma and not semicolon, use CSVEd to change it.
Finally, if your time column is in a different format, try to change the Date Format string. The format should follow the date/time format used by .NET framework. Refer to this article for details on the specifications.
Take note that the generated IPD file has only been tested on Windows and may not work on the Mac version of Blackberry Desktop Software due to file format differences. Unicode text messages are supported, just make sure that your CSV files are saved using UTF8 encoding. As always, remember to backup all data on your phone prior to proceeding.
For those who are interested, MagicBerry is a good utility to explore, split, merge or extract information from the IPD backup file. ipddump is an open-source Java-based tool that can also do the same for IPD files. To extract data from the newer BBB file format generated by Blackberry Link, you will need a paid tool such as Blackberry Backup Extractor.
MD,
I would like to ask about the header.ipd asking for and how to create it?
Hi, the header.ipd file is already in the CSV2IPD.zip file that you downloaded. Or do you want to create the file yourself?
yes I'm trying to convert my CSV file to IPD, but I failed to convert it,there is an error in output ipd file, what should I do?
Hi, any error messages you're getting? If you can send me your CSV file, I may be of help. Take note that the CSV file format has to be exactly as shown in the article.
My email: M D A N H 2002 at G_ma_i_l dot COM
I saved it in the exact same format you have specified and tried to convert, it still says "Done. 1976 invalid record(s) ignored."
Please help.
Hi, can you copy and paste a few lines of your Inbox and Sent Items CSV files here and I will check if they're correct. You can also send me the CSV files, my email address can be found above. Thanks for trying the tool!
The problem is that the delimiters are commas, and the text also contains commas, so the messages are getting split into several columns, i don't know if that makes sense.![:-(]( "Sad")
Hi, despite the name CSV, the delimiters should be semicolons and not commas. If you follow the CSV specs closely and surround each column with double quotes, as indicated in the example, the case of the message text containing the delimiters should be handled properly. Let me know if this helps.
Hi, I had the same problem of the last user: it always says me "Done. 326 invalid record(s) ignored".
The CSV structure is the same you have shown in the examples and I have semicolons as delimiters..
Where I'm failing? Thanks!
Hi Giorgio , can you copy and paste 1 line each of your incoming and outgoing CSV files here? I will try to see what's wrong.
Hi, here are the two lines.
I've only replaced with 10 X the phone number:
sms;deliver;"+39XXXXXXXXXX";"";"";"2011.09.07 17:27";"";"Ciao:)"
sms;submit;"";"+39XXXXXXXXXX";"";"2011.11.16 17:06";"";"Prova"
Hi there, I fixed a potential problem which may result in failure to parse the CSV files on some systems. You can re-download the app and try again. If it still doesn't work, you can send me the CSV2IPD.log found in the app folder and I will help to debug. Thanks!
Hey all. When I export the messages in nokia suite, it exports everything into one .csv and they come up as:
"sms","READ,RECEIVED"
"sms","SENT"
Do I find and replace those with:
sms;deliver;"
sms:submit;"
Hi, yes, the 2 CSV files (1 for Inbox items and another for sent items) need to look exactly as the sample. Let me know if this helps.
Only managed to get one CSV file since it said to export all files and won't give me another option to split up.
Hi, with some help of Notepad/Excel or perhaps CSVEd, you should be able to sort the records and split them into 2 CSV files![:)]( "Smile")
I'll give it a shot.![:)]( "Smile")
If I send you what I have so far, can you tell me if I'm doing it correctly? The message I got when I output was Done. 0 invalid record(s) ignored. Refers to csv2ipd.log for details.
Hi, it seems you are doing it correctly![:)]( "Smile")
Now use Blackberry Desktop Manager to restore the generated IPD file to your Blackberry and check if text messages are migrated. Cheers.
Hi, this is where I have a bit of a problem. Where in the BDM so I restore the IPD file? I tried to do it in memo pad but got an error. Should I do it in the contacts part?
Hi, you may follow this guide
http://btsc.webapps.blackberry.com/btsc/search.do?cmd=displayKC&docType=kc&externalId=KB10339
Thanks, it worked. Some came out double but heck, better than NO messages and I deleted the duplicates and it's perfect. I'm sooooooo ELATED right now! Thank you.
must i install any programs before running this application? is there an exe in the folder?
It requires .NET 2.0 (should already be already installed by default in Windows XP SP3/Vista/Win7). Just run the executable and use the tool to convert.
Hi Blessed New Year,
I have a question, I was using my BB and also nokia phone together. So if now I would like to like put the sms from my nokia phone to merge with my BB phone is that possible?
My bb phone already has sms now on it.
If there is a way how am i to do it?
Hi, I think Blackberry Desktop Manager may be able to merge the SMS in the IPD file with the existing SMS on your device. If in doubt, back up the current messages on your device first before trying.![:)]( "Smile")
Happy New Year to you too!
Hi there
After a few hours of preparing and tweaking my backup file from Android I finally succeeded. Thank you very much for this great tool!
There were two problems which caused me quite a headache:
1) Be sure that your CSV file is UNICODE encoded. You can change this by clicking "Save as.." -> set Encoding to "Unicode" in Windows Editor.
2) The generated IPD file CANNOT be transferred using the MAC version of BlackBerry Desktop Software.
I hope this helps.
Hi Alessandro,
Glad you got it working and thanks for sharing your finding with me. I only tested the generated IPD file on Windows (I don't have a Mac) so I believe it may not work on the MAC version of Blackberry Desktop software due to file format differences. I will put this information in the article. Thanks again!
Hi MD,
I must say this is a great application. Thank you so much for writing this.
Also, I have couple of questions: I'm trying to transfer my smses from Android to Blackberry using your application. I wrote an application to backup all my sms from Android to csv-Unicode encoded file and everything is fine except the content of smses with Vietnamese characters. The only problem is the Byte arrays from Vietnamese Unicode strings somehow cannot be decoded by BDM. However, when I tried to backup vietnamese smses from my Blackberry and restored, they're still fine.
Do you have any ideas about this?
Thank you
Hi Hung,
Thank you for your feedback. I have just tested the application and it indeed had problems with some Unicode characters such as Vietnamese, Chinese or Japanese. I have fixed the problem and update the download as well as the source code. Appreciate if you could re-download the tool, generate the IPD and import it to your Blackberry again. Make sure that your CSV files are saved in UTF8 format. I have included sample text messages in various languages in the sample CSV files in the download, you can refer to these files.
I hope this helps. Let me know if you can successfully imported your Vietnamese text messages![:)]( "Smile")
Awesome, it worked..![:)]( "Smile")
Thank you so much, MD
Also, is there anyway to convert back the ipd data to csv without losing the content? Since I checked most the ipd editor software and they couldn't decode the Vietnamese characters too.
Thanks
Hi Hung, glad it works for you![:)]( "Smile")
Reading Unicode characters from IPD file is difficult as the format is undocumented and therefore many tools such as MagicBerry or ABC Amber BlackBerry Converter fail to handle properly. There is another open source project, http://code.google.com/p/ipddump/, which reads an IPD file and dump the records to a text file such as CSV. You may want to give it a try.
Let me know if you have any other questions.
Wow, the ipddump is awesome.![:D]( "Very Happy")
Thank you so much
Hi MD ,
I have some problems using it , but really i'm very happy for finding such a program .
i sent to you my file so that you can analyze it , i think that my problem may be because that my country number is longer than yous , i don't know … waiting for your feedback "neso_eg@yahoo.com"
Well, having a new Nokia as well, all in one file and just a little bit over 9000 messages in total![:)]( "Smile")
only 45 have been shown up as invalid after split via script.
Just copy & past into a textfile and rename the textfile to split.vbs.
Make sure the script and input file are all in the same folder.
HtH,
Thorsten
—————– Copy & Past everthing below this line into the script file ————-
option explicit
dim objFSO, objAllMsg, objInBox, objOutBox, objTmp
dim strAllMsg, strInBox, strOutBox, strLine, arrTmp, strWrite, strValue, intKey, strMode
strAllMsg = "N95_allMSG.csv" ' replace against the name of your export file
strInBox = "1InBox.csv" ' contains all inbox messages
strOutBox = "1OutBox.csv" ' contains all outbox nessages
set objFSO = CreateObject("Scripting.FileSystemObject")
' make sure, the file is created and that we can write to it!
set objInBox = objFSO.CreateTextFile(strInBox, True)
objInBox.Close
set objInBox = objFSO.OpenTextFile(strInBox, 8, True)
set objOutBox = objFSO.CreateTextFile(strOutBox, True)
objOutBox.Close
set objOutBox = objFSO.OpenTextFile(strOutBox, 8, True)
' read the combined sms log file
set objAllMsg = objFSO.OpenTextFile(strAllMsg, 1)
Do Until objAllMsg.AtEndOfStream
strLine = objAllMsg.ReadLine
' inbox message
if (InStr(strLine, "RECEIVED") > 0) then
set objTmp = objInBox
strMode = ";deliver;"
end if
' outbox message
if (InStr(strLine, "SENT") > 0) then
set objTmp = objOutBox
strMode = ";submit;"
end if
' is the line a new message or further text?
if (InStr(strLine, "sms") = 2) then
arrTmp = Split(strLine, ",")
arrTmp(0) = Replace(arrTmp(0), Chr(34), "")
intKey = 0
for each strValue in arrTmp
if (intKey = 0) then
strWrite = arrTmp(0) & strMode
end if
if (strMode = ";deliver;") then
if (intKey > 2 and intKey < 8) then
strWrite = strWrite & arrTmp(intKey) & ";"
end if
if (intKey > 7) then
strWrite = strWrite & arrTmp(intKey)
end if
end if
if (strMode = ";submit;") then
if (intKey = 3) then
strWrite = strWrite & Chr(34) & Chr(34) & ";" & arrTmp(3) & ";" & Chr(34) & Chr(34) & ";" & arrTmp(6) & ";" & Chr(34) & Chr(34) & ";"
end if
if (intKey > 7) then
strWrite = strWrite & arrTmp(intKey)
end if
end if
intKey = intKey + 1
next
if (InStrRev(strWrite, ";", -1) = Len(strWrite)) then
strWrite = Left(strWrite, Len(strWrite) – 1)
end if
objTmp.Write(strWrite & vbCrLf)
else
objTmp.Write(strLine & vbCrLf)
end if
Loop
objAllMsg.Close
objInBox.Close
objOutBox.Close
WScript.Echo "All Done"
Hi Thorsten,
I am glad it works for you and thanks very much for sharing your VBS script to split the exported file into 2 files. I believe this will help others with similar problem![:)]( "Smile")
Cheers
Good day, Minh! Seems like an awesome program, but I can't seem to manage it. I tried setting the right date, ensuring separators are semi-colons, and the csv is saved as unicode.
Still, all items are returned as errors, no success.
CSVs are exports from Nokia N95.
May I also send my files for you to advise on what I'm missing?
Thanks so much.
I'm at ah me d >>> . >>> ca ti c >>> @ >>> ro ge rs >>> . >>> com
Thank you!
Hi,
I'm unable to convert my csv file to ipd.
Can you please send me your mail id, I will send the csv file.
My id – gthm.nm@gmail.com.
Thanks,
GM
Arabic Unicode supported!![:)]( "Smile")
Thanks in advance
Hi i can't convert it the same way u did, can you please help me do it![:)]( "Smile")
my email : venomousman92@hotmail.com
Please contact me
Thanks for this valuable information. I am new to Blackberry and was needed the same. I was not familiar with this and was just trying to do it, my way. But everything was worthless. But this article is really very helpful.
Thanks again for this great help!!
hi MD, i tried to use your program to convert my csv file to ipd but doesnt work, can u pls teach me how t do it?
Hi MD
This is awesome, but I have one problem. After converting the csv to ipd I'm using MagicBerry to merge the ipd from your app and my backup ipd. When I check the results of the new one, all the phone numbers on my sms are now special characters. Any idea of what I'm doing wrong here. Thanks
Hi,
A few suggestions here
(1) Can you try to restore the original IPD file as created from the CSV2IPD tool to the Blackberry device and see if the SMS are correctly restored? If yes, the problem was with the merging.
(2) With the merged IPD file being restored, do you have problems only with the SMS phone numbers (I assumed you meant the number of the SMS senders) or with the SMS body text as well? Are the SMS timestamps correct?
My guess would be that the IPD file generated from my tool has the file format of an older version of Blackberry Desktop Manager whereas your backup file has the latest file format. And for this reason, MagicBerry doesn't handle the merging well, resulting in your mentioned problem. Also, MagicBerry is a bit buggy and would have issues handling SMS with Unicode characters.
You can try the following, let me know if it helps
(1) Back up all your current SMS into an IPD file and keep it safe
(2) Restore the IPD file generated by the CSV2IPD tool onto the device. Check if the SMS are restored properly.
(3) Use Blackberry Desktop Software to backup the newly-restored SMSes into another IPD file.
(4) Use MagicBerry to merge the IPD file in step (3) with the IPD file in step (1)
(5) Restore the merge IPD file onto the device
If the file format is indeed a problem, performing the above would most likely help as the 2 IPD files are both generated by Blackberry Desktop Software and MagicBerry hopefully will merge it properly.
If it does not help, you can try using a tool such as ipddump to dump you current IPD file into CSV file format, merging this CSV with your original CSV, and use CSV2IPD again to create a single IPD file to be restored to the device.
Let me know how it goes. And thanks for your feedback too![:)]( "Smile")
Thanks MD, this sounds good, but I think the problem I'm having is with the MagicBerry. Like you said maybe this software have issues handling SMS with Unicode characters. The only problem I had was the sms phone number.
Anyways, my phone is taking your file with no issue using the blackberry desktop manager. Thanks again for your help and for this incredible tool.
Hi, MD
i have a single .csv which contains both incoming and outgoing messages, but the software does not recognize it. creating separate files will take a lot of time. is there any other way for doing it.
Hi, if you can send me the CSV file I will be able to help convert it to an IPD file for you.
Hi, I have a problem with importing IPD file by Blackberry Link. Everything seems to be fine but there's noe effect on the phone (Z10). I also tried to conver IPD into BBB but there's still problem with importing those SMS. Do you have any idea what could be wrong?
Hi Szymon,
DId the CSV2IPD tool report any errors during the conversion to IPD file? How many SMS were reported as being successfully converted?
Did you use Blackberry Desktop Software to import the IPD file? What exact problem do you have when importing? Try to use Blackberry Desktop Software and see if it can recognize any SMS inside the converted IPD file.
However, take note that the tool has only been tested with Blackberry OS 4/5/6 or 7 and have not yet been tested with Blackberry OS 10 devices such as the Z10. Look forward to your feedback so that I can improve the tool and make it work with OS 10 devices.