There are no hard-and-fast rules for the best way of converting or migrating data between platforms, applications or formats: each case must be examined on its own merits. Typical considerations are:
- Data volume
- Storage mechanism (relational or other database, flat file, application-specific etc.)
- Data complexity (flat records, BLOBs, structures etc.)
- Binary formats (dates, floating-point, zoned / overpunched / packed etc.)
The two principal methods used by RapidGen are:
- A custom record-by-record conversion program, auto-generated by the Genius Transfer Utility. This works well for conversion of flat files between different record layouts stored in the Genius Data Dictionary. Variables can be auto-converted (e.g. from VAX floating to IEEE floating) at the same time. This method can also be used for certain types of database, but may not be the most efficient approach for large data volumes.
- Unloading to serial format (XML, CSV etc.), performing any necessary conversion or reformatting, then loading into the new system. This is usually the best approach for converting from one type of relational database to another, using the inbuilt backup/.restore facilities which are optimized for the specific database type. Conversion between disparate save/restore formats can usually be done using XSLT if the source format is XML. In other cases it is usually necessary to create a custom conversion program, either manually or by defining the serial formats to the Genius Data Dictionary and using the Transfer Utility as above.
In more complex cases a combination of these and other techniques may be necessary, particularly when data volumes are large. Additional complications can include:
- Character set conversion (EBCDIC, Unicode, Windows code pages etc.)
- Little-endian to big-endian numeric conversions and vice versa
- Data dependencies and database constraints requiring strict load sequencing, partial or split loads or constraint deferment
- Graphics and multimedia format conversions
Conversion programs can usually be written (manually or using the Transfer Utility) in RapidGen’s RPL language, which employs Decision Table logic and natively supports an extremely wide range of data types. In some situations, however, other languages such as C++ or FORTRAN may be more appropriate; RapidGen can advise on request.
RapidGen is adept at developing ‘Bespoke Solutions’, tailored to meet Client requirements and has an excellent record of resolving complex challenges of ‘Software Plumbing’