Streamlined information integration and governance
Gijs Logman, Database Administrator, says: “Instead of cloning the full production databases into our test environments, we decided to extract a much smaller subset of records from each database, which could be copied much more quickly.
“However, that is not as easy as it sounds. In a relational database such as Oracle, when you copy a record from one table, you also need to ensure it is referentially intact—which means that for any parent row, you also need to copy all of its related child rows. Otherwise your dataset will lack integrity and your test applications may not work properly. When you have a large, complex database with hundreds of tables, it’s almost impossible to select all the data you need manually. That’s why we decided to use InfoSphere Optim software from IBM.”
IBM InfoSphere Optim Test Data Management is a tool that allows database administrators to define a custom data model that maps both explicit and implicit relationships in a source database, and set sophisticated business rules to select an appropriate subset of records from it.
The solution then traverses the database schema and extracts all the appropriate records, together with their foreign key relationships and any other related data specified in the Optim data model—ensuring that each business object is referentially intact. Once the solution has been configured, it can be scheduled to run automatically, with no need for further manual intervention from the database team.
Frans Verduyn Lunel comments: “We started out by setting Optim to select every hundredth record in our customer account database, but over time we have begun to use it in more sophisticated ways. For example, we can specify that we want customers who have more than a certain number of transactions, or who have been active within a particular date range. The ability to add selection criteria to the data model is very powerful, and it is based on SQL syntax so it’s easy for most database administrators to use.”
Gijs Logman adds: “We’ve been using Optim for ten years with our Oracle databases, and we have never experienced any problems with it. It’s very reliable and we’re completely confident in the integrity of the datasets it generates.”
Giving developers the data they need
Thanks to the IBM solution, BinckBank is now able to give its development teams fresh copies of a realistic subset of production data on a weekly basis. The process runs automatically over the weekend, and in less than a day, all 13 test environments and 52 Oracle test databases are seamlessly updated, with no need for manual intervention from the DBA team.
Frans Verduyn Lunel comments: “We’re able to provide a dataset that has very similar characteristics to our production data, but is only ten percent of the size. That means we require 90 percent less storage capacity in each of our test environments, and we don’t need such powerful Oracle database servers—so the cost savings are very significant.”
Gijs Logman says: “Refreshing the test environment every week means that our developers always have realistic, up-to-date data to test their code against. This would be impossible without a tool like InfoSphere Optim, because we would never be able to take a full copy of the production databases within the weekend maintenance window, and creating a subset manually would be extremely time-consuming and difficult.”
Frans Verduyn Lunel concludes: “Above all, Optim gives us confidence that our test datasets have complete referential integrity, so the testing process runs smoothly. If we identify a defect in testing, we know it’s the code, not the data that is at fault. This helps to maintain high levels of quality and speed in our development process, and deliver innovative products and features to our customers ahead of the competition.”