Christopher Shennan's Blog

A day in the life of…

I am often importing data into a symfony project from an existing site or a CSV file and re-writing this data into the fixtures.yml is usually too complicated or time consuming so I end up writing bespoke import actions.

While I am writing these actions I often find that I have to delete the contents of the MySQL tables as you can never write the routine complete and accurate in one go so as you build it up in smaller steps you find that you have to get rid of the previously old data.

This presented me with 2 problems:-

  • Using DELETE FROM [TABLENAME] does not reset the auto-incrementing fields so if you have anything that is based on the id during the rest of the import then you’re going to have issues.
  • There is a lot of clicking in phpMyAdmin to empty the tables, especially if you forget about the foreign key relationships.

I found that MySQL had a TRUNCATE TABLE but this is not directly available via Doctrine as the TRUNCATE TABLE is function of the type of database you are using and Doctrine cannot assume you are always using MySQL so therefore it cannot provide a generic method to call it.

Having said that it was not too difficult to run the TRUNCATE TABLE from within my Symfony project.  All I had to do was add the following lines of code to an action within the actions.class.php for my import module and now it empties each table I have defined before importing the data and it also resets the auto-incrememt ids all my tables start from id = 1 each time I run the import.

$dbh = Doctrine_Manager::getInstance()->getCurrentConnection()->getDbh();
$dbh->query('TRUNCATE TABLE order_items');
$dbh->query('TRUNCATE TABLE orders');
unset($dbh);

Be careful which order you run the TRUNCATE TABLE calls so you avoid the foreign key relationship issues.

Let me know if this helps you out.  Enjoy!

About Christopher Shennan

I am a web developer specialising in web driven applications using PHP, MySQL, Symfony and Zend and I am currently working for Line Digital in Edinburgh, Scotland.

Most days I can be found frantically coding away with EuroDance in my ears and consuming what I hope to be a never ending supply of coffee... happy days!

Connect with me via Twitter, Google+ or LinkedIn

  • #1 | Written by Ivar about 2 years ago.

    Thank you.

    Trunkating all tables in phpMyAdmin was driving me crazy. I saved myself some typing by separating the query with ;

    TRUNCATE TABLE Article; TRUNCATE TABLE User; …

  • #2 | Written by Parag about 2 years ago.

    Thanks a lot! I don’t know where this is in the Doctrine docs, but your solution is exactly what i was looking for. Can we use DROP and ALTER table statements in the same way?

  • #3 | Written by Christopher Shennan about 2 years ago.

    I don’t see why not although I think DROP and ALTER table statements should really be handled with migrations.

    I’ve started using these recently and although I’ve found there are a few things you need to remember when using them they work really well and really reduces the problems when applying database changes from development to test to live servers.

  • #4 | Written by jbreuer about 2 years ago.

    What you get with:

    $doctrine = Doctrine_Manager::getInstance()->getCurrentConnection()->getDbh();

    is a PDO-Object (in most cases), so the naming of the variable ($doctrine) could lead to confussion.

  • #5 | Written by Christopher Shennan about 2 years ago.

    Hi jbreuer,

    Thank you for the clarification. I’ve updated $doctrine to $dbh to help avoid confusion with this example.

    Many Thanks.

    Chris Shennan

  • #6 | Written by Ariaan Bruinsma about 1 year ago.

    Isn’t this sufficiant?

    ./symfony doctrine:build –all –and-load; ./symfony cc

    This way you only have to maintain the schema.yml and fixtures.yml.

    Or didn’t I understand the main point?

  • #7 | Written by Christopher Shennan about 1 year ago.

    Hi Ariaan

    The problem with ./symfony doctrine:build –all –and-load; ./symfony cc is that it is completely nukes the database and loads everything from scratch so you have to have fixtures defined for all your data. In my case I was working on importing data for a few tables and someone else was working on other parts of the system so this was not appropriate.

    Using truncate in the way I have mentioned meant I could get my data import tasks to only clean out the tables which I needed to work with whilst leaving the rest of the system intact, usable by the other person and avoiding possible loss of data due to records not being in the fixtures.yml (added or edited via the administration pages but not converted back in to the fixtures.yml)

    In addition, the trouble with doing all the data in fixtures.yml is that it is not particularly client friendly. For a developer, it is probably no harder to maintain the fixtures.yml or a equivalent CSV file but I find that the client typically prefers something they can easily visualise, hence why I was importing using a CSV file (which the client was constantly updating and I was re-importing) rather than using a fixtures.yml. That said, there is nothing wrong with using a fixtures.yml to do the same job.

    I hope this clears up why I was using truncate and a CSV in this manner.

    Many Thanks.

    Chris

  • #8 | Written by Christian about 6 months ago.

    Doctrine::getTable(‘OrderItems’)->findAll()->delete() ?

  • #9 | Written by Christopher Shennan about 5 months ago.

    Hi Christian,

    That would delete all the records and should take care of removing the related tables too provided the relationship is CASCADE DELETE and not CASCADE SET NULL. If the relationship is CASCADE SET NULL then you are leaving redundant data behind which may be perfectly valid in normal operation but probably not while repeatedly running the import scripts.

    Unfortunately that way wouldn’t result in the index being reset which may not always matter, but if as per my first point you have id specific import logic then subsequent attempts would likely not work as anticipated.

    Many thanks.

    Chris.

No trackbacks yet.

Leave a Comment

Subscribe to comments

CommentLuv badge