Technical Overview

Document Content

The Databases

There have been 7 database systems into which CPR data have been put. The first, based on COBOL programs and files, ran on the Manchester Computer Centre main computer and was single user only. The second, based on the Ingres Vision product ran on the Liverpool Computer Science HP-UX network. The third, was a port of the Liverpool System to the then new MIDAS computer in Manchester. The fourth (called CPRSDB) was a reload following the upgrade of Ingres software on MIDAS to OpenIngres. This system was tentatively made available over the Internet for a brief period in 1998. CPRSDB was withdrawn because of problems with the translated COBOL datasets. The current system (called CPRdb) is based on data loaded from the original source files using extensive data validating and checking techniques. These data were loaded into OpenIngres Version 2.0 on the MIMAS computer in Manchester. (MIMAS was the new name for the MIDAS service). To identify CPRdb software (initially the data loading system) a versioning system was started at "CPRdb v1.0". Frame based web access became available with CPRdb v2.0 and with v2.1, three types of data access was possible. When the MIMAS OpenIngres service was withdrawn in 2003 CPRdb was moved to a SQLserver environment at the University of Manchester. Version 3.0 of CPRdb (the 7th actual database system used) contained the same data and features as CPRdb v2.1 but transferred to an Oracle database system at the University of Liverpool Computing Services. Subsequent versions of the system (currently at CPRdb v3.6) have, since Spring 2004, been hosted by the Liverpool Computer Science Department's Oracle and Web services.

The database available on the Internet for general access is called CPRdb and was officially launched on 12th April 2001. CPRdb contains all the data currently loaded - see Progress pages elsewhere for details. The database is updated in two ways. Bulk data loading, is an involved multi-stage process that loads an entire parish event or events. There is much data validation, consistency, and other checking done in these procedures. Minor corrections and update, can be done at any stage by authorised database editors.

In due course, more advanced access facilities (based on Portal Architecture) will be made available using the Liverpool Oracle server. The intention is to make use of OLE like access strategies allowing the database to be used in a seamless way in multi-database projects.

Internet Access

Internet access to the Oracle 12c DBMS server is provided by use of CGI scripts (CPRdb v3.0, v3.1, v3.2, v3.3, v3.4) written in PERL and (CPRdb v3.6) JDBC modules. The software versions implement new facilities, or correct bugs. Three methods of access are currently under investigation:

Query by Form Filling (QFF), is currently the only method casual users may use to get to CPRdb data.

Query By Template (QBT), is being examined as an alternative to FF. However, there remain some security questions concerning this mode of access and until these are resolved access to QBT is restricted.

A third method, SQL Command Entry (QCE) is available only to those with UserID and PassWord access to CPRdb. This method allows users complete SQL access to the entire database, and allows them to enter complex multi-table queries, etc. If the security issues associated with the QCE access method can be resolved, then this method (perhaps in a limited `select' only mode) will be made more generally available.

A survey of user preferences in 1998, suggested that for casual use QFF met all requirements. However, advanced users preferred QCE despite the heavy learning curve associated with the method. We hope to respond to these findings, as soon as we can satisfy ourselves concerning security and integrity of CPRdb as a whole.

Data Loading

There is extensive documentation concerning data validation methodology and data loading, however, this information is available locally only. Suffice it to say, that we believe our methods guarantee that what the transcribers recorded is made available to users/consumers.

Development Team

A very large team has been, and continues to be involved in bringing CPRdb to you. Mostly, they are students at the University of Liverpool undertaking BSc and MSc projects under the supervision of Dr Bernard Diaz (Technical Director of the CPR Project). However, many people have been involved and it gives us great pleasure to record their names here and to thank them for their contribution.

Trevor J Howkins
University of Manchester
1979Designed and implemented the Apple micro-computer data entry system.
M Anne Maybrey
University of Liverpool
1980Supervised the design and implementation of the COBOL based system
Robert F Cuthbertson
University of Manchester, MSc Student
1982Designed and implemented the extended COBOL based system
John C Clegg
Manchester Computing Centre
1982Principal Programmer on the COBOL systems. Ported data & software twice ultimately to the Manchester Amdahl. Converted the Apple data entry and editing suite to run on BBC micro-computers
Alan Quayle 1992Converted the BBC programs to run on PC systems
Bernard M Diaz
University of Liverpool
1993Involvement began when he asked "...if there were any Warrington data ?" Supervised the design and implementation of CPRdb
Tom Johnston
University of Liverpool, BSc Student
1993Implemented the first nominal data visualisation system, using data provided by Ann Maybrey
Simon J Kirkham
University of Liverpool, BSc Student
1994Designed and implemented the Liverpool Ingres system. Converted COBOL datasets to Ingres. Wrote the Ingres data load system. Implemented the Ingres Vision pro-am access system. Ported data and system to MIDAS. Implemented the first MING (MIDAS Ingres) period system
John F Fothergill1996Data conversion system BBC to PC format and update of the PC edit suite. Principal data validator.
Yee Mei (May) Kwok
University of Liverpool, MSc Student
1998Interface flavours - QFF, QBT, QCE. Organised Interface Assessment Weekend.
Michael J Staniforth
University of Liverpool, BSc Student
1999Use of Java Servlets
Fiona C Silk
University of Liverpool, BSc Student
2000Tagging and notes system
Clair Fiona G Knowles
University of Liverpool, MSc Student
2000Web based CPRdb Update
Brian J Bilbao
University of Liverpool, BSc Student
2001CPRdb web based statistics
Ricardo I Delandro
University of Liverpool, BSc Student
2001CPRdb update using PHP
Kieron M Dye
University of Liverpool, BSc Student
2001Flexible CPRdb update
Stephen IM Murray
University of Liverpool, BSc Student
2001Tools to support tagging
Daniel J Williams
University of Liverpool, BSc Student
2001Tagged QFF

Revision History
BMD - 9 September 2009
BMD - 16 December 2003
BMD - 11 April 2001.