IT systems advanced rapidly in the 1970s and 1980s. Numerous hardware vendors entered the market, and the number of different formats rapidly multiplied as each vendor developed new techniques to store and access data faster by advantage of their own hardware. At this time, vendors saw proprietary data formats as a technical competitive advantage and as a strategic way to lock-in their customers.
While some data storage methods faded into obscurity, other prospered because of their technical advantages and commercial success. But by the mid-1990s, relational databases emerged as the dominant data storage technique for applications with only minor differences in SQL implementations between vendors.
RapidGen has supported the full range of data formats in common use since the 1970s, including both relational and non-relational databases. It is interesting to note the current trend to promote “NoSQL” and “BigData” as innovative for using non-relational databases, but RapidGen has supported non-relational databases for 30 years.
The majority of legacy systems stored data in a variety of proprietary formats and structures that are not easily accessible by contemporary software. In the 1980s, standards like ASCII, Ethernet, FTP, JSON, ODBC, SQL, TCP/IP, and XML had not become established. Instead, vendors created their own proprietary and incompatible formats.
One example of proprietary and incompatible formats that routinely impacts legacy systems is data types. Different legacy hardware architectures, operating systems, and programming language use different methods for physically storing data. Examples include
- Signed, unsigned, scaled, and byte-swapped integers
- IEEE, VAX (F, D, G & H) and RSTS (byte-reversed VAX) floating-point formats
- Zoned and overpunched numeric strings
- Binary dates and automatic century deduction
Supported Data Sources
The majority of data held in legacy systems including
- Applications including CODA-IAS, Cyborg, Filetab-based applications, Protos, and Tropos
- Hierarchical and Codasyl Databases including DBMS-32
- Indexed Flat File Structures including RMS, C-ISAM, D-ISAM, and MicroFocus Cobol Structures with full control of locking
- NoSQL Databases
- Relational Databases including Oracle, Oracle Rdb, CA-DB, Ingres, and generic ODBC.
- Spreadsheets including Lotus, Excel, and 20-20
Unknown Data Formats
RapidGen’s expertise and software suite have been applied to virtually every legacy data format ever used. However, the RapidGen tools also allow our consultants to access unknown data formats even if generic drivers such as ODBC cannot be used and the database interfaces are undocumented.
For advice or assistance with your data sources, please let us know how we can help.