Entrepreneur Geek

We use Paypal to receive payments from customers. We maintain our financial accounts using Tally. Imagine copying over Paypal transactions into a separate accounting software – manually. You are creating all ledger accounts one by one. Ensuring spellings are correct, accounting heads are right and you are not missing any names in between. You’re numb by the time you’ve created 500th account, but still have another 500 to go. You drag yourself through the process. Over the next few days, you create sales and receipt vouchers for each sale. But hey, did you forget that the sales are in US Dollars, but income tax returns have to be filed in Indian Rupees? You now convert all amounts to suitable Rupee values. You probably use some Excel formulas to make it easier, but you still need to get those entries in Tally. Few more days of work, uh?

Urgh… It would be a nightmare to do this manually

This is the first financial year where we are submitting our returns, and I wanted to make sure everything is impeccable. My wife has extensive experience with Tally and she told me she could quickly make these entries.

I laughed, I knew it was not so easy, and that she was over optimistic!

There is no way this is done manually, I said to myself.

Lazy Programmers Automate Things

Being a lazy programmer, I wanted to automate this. Automating mundane tasks like this has numerous advantage. First, we will save a lot of data entry time (and headache). Second, we will save this time and headache every time in future we wanted to bring our accounts up to date. Third, it will avoid data entry errors. And forth, I will get to learn a bit more of Tally.

(To give you a bit of background on this, I have graduated in Commerce with Bookkeeping as my main subject, but I don’t remember any of it now. I always liked programming better! At the same time, I pity the accountants, seeing them working doubly hard just copying data from one source to another.)

It was my long time dream to import data in Tally. I was always told it’s possible to export from Tally to Excel, but not vice versa.

Time to break the myths!

Strategy 1: Import Excel Data to Tally

I looks around the web to find if someone else had similar problems and came up with some solution. And sure, I found UDI Magic by RTSLink.

RTSLink creates excellent software to import Excel to Tally, SQL Server / MySQL / DBF and virtually any database to Tally. The have a free utility called UDI Magic that allows importing Excel data to Tally.

I felt ecstatic. I found my solution so quickly! The next day, I downloaded UDI Magic and gave it a spin.

I found it worked really well. I could create ledger masters, sales transactions, stock transactions and a lot more with this software. It takes Excel data, converts it to XML and imports it to Tally using RTSLink DLL.

I decided I will convert my Paypal data to CSV. From CSV to Excel in a format UDI Magic can understand. And then to Tally.

This was a perfect solution. But had two problems.

1. I will still need to convert Paypal data into a format UDI Magic could understand
2. UDI Magic is a trial software. It works for 30 times / 30 days and expires.

Now I could keep installing it on different machines whenever I needed or try some other hackery to get around the expiry limitation. But I said, heck, why not be clean? Why not push my conversion system a bit further and convert to Tally XML itself?

Strategy 2: Solve the problem once and for all – Write a script to convert Paypal data (residing in MySQL) to UDI XML for Tally

I thought it would be easy. I know PHP and XML like the back of my hand. I can put together a converter in an hour.

Alas. It was not that quick. It took me 8 hours to write the converter. It still does not handle Purchases (I got tired, and did not have so many purchase transactions). But at the end of the day, I was able to import close to 600 master ledgers and about 900 vouchers in Tally in under a minute. That was sweet!

So how does this work?

Here is an overview

1. Login to Paypal. Go to History. Download CSV reports – along with shopping cart details. If it takes too long to download full year’s transactions, download monthly batches.
2. Download PaypalToTally.zip – unzip the contents to your local web server directory
3. Create paypal_history table in your MySQL DB using paypal_history.sql file.
4. Import Paypal transactions to this table using phpMyAdmin – Import Data.
5. Ensure data is imported correctly. Fix problems if any.
6. Configure database details and other configuration options in config.inc.php
7. Make sure “data” folder is writeable (chmod 666)
8. Hit the index.php via browser!
9. Ensure generated XML files are valid. (You can open them with browser)
10. Open Tally. Open the company you want to import to.
11. Import Ledgers using Import Masters option under “Gateway of Tally”
12. Verify it worked.
13. Import transaction vouchers using “Import Vouchers” option
14. Verify it worked!
15. Enjoy!

Your Mileage May Vary

I wrote these scripts to solve a specific problem – import sales data from our Paypal account to Tally. Such that Tally shows all customers, their payments and products purchased. You will need to modify these scripts to suit your needs. These scripts support Web Accept Payments, Shopping Cart Payments and won’t record Refunded items. But they won’t import Purchases or simple Payments Received  / Sent (via Send Money tab in Paypal).

You can modify these scripts to import data from any database. If you add any features, do let me know!

Common Problems / Troubleshooting

• Ensure the financial year is set correctly. Dates in import files must be within current financial year in Tally.
• Make sure starting voucher number is set correctly. Vouchers won’t import otherwise.
• If only some of the data gets imported (Tally quits importing at certain percentage), check the imported data. Find out last imported transaction. Locate it in the XML file. You have some problem with the transaction after this.
• These scripts don’t import Purchase information yet.

Free Download

In a hurry? Download PaypalToTally.zip from here.

References

• RTS Link website was extremely helpful. Especially their explanation of Sales vouchers XML and Receipts XML.
• You may need a Tally expert around. My wife cleared some of my confusions about Debit and Credits

Using Paypal? You may like Putler

Do you use Paypal for your business? Then you may even like Putler – a Paypal Reporting, Monitoring and Planning tool.

Thank you, share your experiences

I hope these scripts prove useful to you. Thank you for trying them out! They saved us many days of work. If you like them, post a comment with your experience!

I am using Text_Diff classes of PHP to generate differences between two XML documents. The output is not always valid XML – tag nesting is not always correct. This happens because my source files are XML and have their own tags. When Text_Diff inserts its own <ins> and <del> tags around the changed text, it messes up the tag hierarchy at times.

I am looking for a clean, fast and safe way to fix such invalid XML. Do you have any recommendations?

I have looked at Tidy, it’s PHP library and htmLawed. I liked htmLawed since it’s pure PHP implementation, but don’t know how fast it is compared to Tidy. Moreover, I need an XML cleaner, not necessarily XHTML cleaner. So even if I use these libraries, I will have to strip out the HTML parts from the output.

Do you have any suggestions / recommendations?

If you are a PHP developer, you probably know that you can type “php.net/substr” to look at documentation for the substr() function. This works for many other keywords as well and is a superb feature. It’s just so convenient, I stopped using my local copy of PHP manual.

But yesterday, I wanted it offline. I downloaded the “many HTML files” version of the documentation, extracted it and set it up on my local Apache. I can now open up http://localhost/phpman/ to read the documentation. (this is how I actually learned PHP 11 years ago!!)

And I wanted the smart lookups – auto complete! I searched around to find if there was a ready .htaccess file that I could use. Or if someone had made something similar already. I did not really find anything solid. Suddenly I realized I can just look up the PHP.net site source code and figure out what they are doing, and replicate it on my local setup. (Yes, PHP.net’s source code is open too!)

Eureka!

So after a few minutes of hacking around, I have my nice little setup that mimics PHP.net’s quick / smart lookup of functions. Typing http://localhost/phpman/substr takes me to http://localhost/phpman/function.substr.html in a split second.

Jai ho!

Let me jump to the code now that you’ve read so much!

Here’s the .htaccess file. And here’s the PHP file – “manual-lookup.php” placed in the phpman folder. The file is an adaptation from php.net website’s source.

Here’s the complete procedure:

1. Ensure you have Apache / PHP setup locally.
2. Download the many HTML files version of PHP documentation.
3. Extract it in a folder called “phpman” in your website root. You should now be able to see it at http://localhost/phpman
4. Download this file, and save it as .htaccess within the phpman folder.
5. Download this file, and save it as manual-lookup.php within the phpman folder.
6. Try http://localhost/phpman/substr – it should work if all went well!
7. Enjoy!

Hope this helps someone like me

I have been writing PHP code for 10 years now and have used a variety of ways to debug my code – except the actual runtime debugging! I have used echo statements and log files to debug and solve problems. I knew there is XDebug and Zend Debugger, but never tried them.

Today, I downloaded, setup and used Zend Debugger for the first time. And I loved it! It was awesome to see variable watch, breakpoints and the like. Just loved it!

Why did I use the debugger? I’ve been using Flex Builder myself for last one year heavily and think the debugger is indispensable. I have done some .NET code earlier and love debugger there too. For PHP, it was more like “I don’t need a debugger!” so far.

I dropped that today and embraced a debugger. And I am loving it so far!

Do you use a PHP Debugger?

Unicode characters and webservices always create one or the other problem for me Working on PlannerX backend was not any different. Spent some good hours fixing Unicode / UTF-8 related issues.

And while I was searching for some solutions, I found an excellent “PHP UTF-8 Cheat Sheet” by Nick Nettleton of DropSend. I highly recommend it if you are going to do anything with PHP and Unicode!

And BTW, don’t use Base64 encoding with UTF-8. It will not work!