| Table of Contents |
|---|
3. Upload and manage affine table and splice interval table files
3.1. Confirm file format and content
Before attempting upload of an affine or splice file using the SCORS Uploader and Manager application, ensure that it complies with the specified format (see Appendix 1 and 2, or recent examples from the LIMS database). The Correlator application should be generating the correct formats. The file must be CSV format and have the prescribed number and types of columns. The uploader will perform a limited number of tests and will reject non-compliant files.
3.2. Load affine
To load an affine table:
- Select the Load affine tab from among the available tabs (Fig. 3-1):
- Load affine (will create a new CCSF depth scale)
- Load splice (requires associated affine to be loaded already)
- Manage (e.g., remove redundant affine/depth scales and splices)
Figure 3-1. SCORS Uploader user interface, Load affine tab.
- If the validation fails, you will get an error message that should be very explicit in terms of type and location of error. The most common errors encountered in initial uploads were:
- If the validation passes, a blinking message will appear: Affine validation in progress, please wait.
- During the validation, the field Name generated from content will be populated; it includes the sites(s) and hole(s) present in the file you loaded.
- The Current date (YYYYMMDD) field will also be populated.
- If validation fails, the message Validation failed, upload aborted, check the affine file format and content will appear. Click OK and check your file.
- If validation is successful, a new blinking message appears: Affine table file upload to asset manager in progress – please wait. Once this message disappears, you can proceed.
- If you want to add a new affine and associated CCSF depth scale to the data base without replacing an existing one, keep the default Add a New Record in the choice list.
- If you want to replace an existing affine and associated CCSF depth scale, select the appropriate name from the choice list. The affine table and associated depth scale will be removed from the database. This action will reduce the accumulation of outdated information, user confusion, and the need for later cleanup in the database. However, consider that in some cases, it may be desirable to keep an older version (particularly if update is provided postcruise) because investigators may have been working with it.
- A blinking message will appear: Affine transfer and CCSF depth scale creation in progress
- The program will display a message when upload is successfully completed.
To confirm successful upload, go to the LIMS reports described below.
The uploaded affine files are stored and accessible in exactly the way they were uploaded. In addition, the affine tables are converted into Oracle database tables such that the cumulative offsets can be used to compute the alternate CCSF depths for samples and results. For any report available within the LIMS Reports framework, users can select the appropriate CCSF scale from the alternate depth choice list. At that point, the CCSF depths are reported for each observation or sample in the report, provided the associated core was included in the affine table to begin with.
3.3. Upload splice tables
To load a splice interval table:
- Select the Load splice tab from the three tabs at your disposal (Fig. 3-2).
- Follow the same procedure as described for loading an affine. The following two steps are unique to loading a splice.
- On the line Select affine for this splice, select the affine that applies to the splice you are uploading. Failure to select the correct affine will certainly produce unexpected results.
- On the line Select optional splice tie point table file you can optionally browse to your splice tie point table (STPT), click the Load STPT button, which will load the file along with the splice interval table file. The STPT will not be used in splice calculations.
The uploaded splice files are stored and accessible in exactly the way they were uploaded. In addition, the splice interval tables are converted into Oracle database tables such that can be used to provide data by splice.
A splice was associated with an affine table at the time of upload and the corresponding CCSF scale is automatically set when user selects a splice from a choice list in the LIMS Report interface.
Figure 3-2. SCORS Uploader user interface, Load splice tab.
3.4. Manage affine tables (CCSF scales) and splices
The Manager tab allows authorized users to delete redundant affine (and associated depth scales) and splices from the database. Unauthorized users cannot use this tab.
If a user cancels an affine table by accident, it cannot be re-instated through the tools provided. However, a developer or database administrator can still recover the original upload files if needed (for a period of at least one year after).
To cancel an affine and/or splice:
- Select the Manager tab (Fig. 3-3)
- Select the affine or splice you want to cancel and click the Cancel affine or the Cancel splice button.
- Note: If a splice has been associated with the affine at some point, the splice has to be canceled first. This is to ensure that “orphaned splices” do not exist in the database.
- If you are canceling a splice, consider if you want to cancel the associated affine/CCSF scale as well (in case it was updated). If so, select the affine you want to cancel and click the Cancel affine button.
Figure 3-3. SCORS Uploader user interface, Manager tab.
4. Optional: Check/fix the splice interval file
4.1. Purpose
The splice interval table (SIT) files generated by the Stratigraphic Correlation Specialist (SCS) using third party correlation applications of their choice may contain information that is inconsistent with the sample registry data in LIMS. Uploading inconsistent information to the LIMS would create unexpected results further downstream, particularly when LIMS retrieves data by splice, or when investigators use the splice table to sample cores. Correlator v. 1 had this problem because it computed section and splice interval boundaries from the data. Correlator v.2 and later should not have such problems anymore and this application is not needed when using that application. However, since the JRSO cannot control what application the SCS use, and how those applications compute depths and section-offset identities, we keep the Splice File Fixer, available to help create splice intervals that are consistent in terms of depth and section offsets.
The Splice File Fixer creates two CSV files for each splice input file:
- A difference report file that has the same name as the input file plus the name extension *DIFF. This file has two rows for each input row (splice interval):
- INPUT line with the same numbers as read from the input file; it is followed by the
- OUTPUT line with corrected numbers based on the logic described below.
- A corrected splice file that has the same file name as the input file plus a name extension *CORR. It has exactly the same columns as the input file and is the file to be uploaded to LIMS if the *DIFF file doesn’t contain significant issues.
4.2. Run Splice File Fixer
Before running the Splice File Fixer, make sure the affine table was loaded to the LIMS because the CCSF scale must have been created in the LIMS before a splice can be tested and corrected. Ensure that the SIT complies with the specified format (Appendix 2). The file must be CSV format and have the prescribed number and types of columns.
Figure 4-1. SpliceFileFixer user interface.
To run the SpliceFileFixer (Fig. 4-1):
- On the Select splice interval table input file line, click the Browse button and navigate to the input SIT file in your directory.
- Click Open/Load in your navigator
- On the Associate existing affine table (CCSF depth scale) line, select the scale corresponding to your SIT. If you don’t see the scale corresponding to a recently uploaded affine table, click the Refresh Affine button.
- If you want to change the threshold value for reporting differences between input and output offset and depth values, set it in the Differences will be reported if they are greater than line.
- Click Validate Input File, and Generate Corrected File and Difference Report button.
- You will see a log of processing steps, with the final reading: “The splice interval table processing has ended or has completed.”
You can now analyze the resulting *DIFF file and upload the *CORR file if satisfied.
4.3. The logic
The Splice File Fixer logic assumes that users trust their selection of core identity (hole-core-section) and CCSF depths for the splice intervals, but may not be certain about the computations of corresponding CSF-A depths and section offsets. The following is the approximate sequence of tests and computations performed by the Splice File Fixer application.
- Create a copy of the input file. Floating point values for offset and depth values are rounded to 1 mm precision fixed values to avoid issues with very small floating-point number differences in depth comparison tests.
- For each splice interval:
- Get all the sections for the given core. Each section record in LIMS has a stored CSF-A as well as CCSF depth (based on loaded affine table) for both top and bottom of section and the array of these four values per section will be used here.
- Proceed with the following steps separately for top and bottom of the splice interval, respectively.
- Find the section that contains the given CCSF depth.
- If the CCSF for the given core is smaller than the smallest section top depth, return message “ABOVECORE” in the *DIFF output file. This will fail splice file upload and user needs to address the issue somehow.
- If the CCSF for the given core is greater than the largest section bottom depth, return message “BELOWCORE” in the *DIFF output file. This will fail splice file upload and user needs to address the issue somehow.
- If two section IDs are returned because the CCSF corresponds exactly to the bottom of a section and the top of the subsequent section, select the section with the smaller section number.
- For the section determined to be the correct one, compute
offset = CCSF(given) – CCSF(section_top).
- Then compute CSF-A = CSF-A(section_top) + offset
- Print the half-line to the output file. Repeat with the bottom of interval if this was the top.
- Proceed with the next splice interval.
- Read the discrepancy threshold value provided by the user (default: 1 cm; Fig. 4-1).
- If discrepancies in section offset and CSF-A depths are larger than the threshold value they will be considered significant and will prompt a difference report, consisting of the OUTPUT line printed below the INPUT line.
- The file has the same name as the input SIT file plus the extension *DIFF.
- The file allows users to review potential issues quickly and take action if needed. A corresponding screen view is also created.
- If differences in section offset and CSF-A depths are equal to or smaller than the threshold value they will be considered insignificant and are not reported.
- Print the output file *.DIFF.
- Print the DIFF report on the screen, in the application user interface, as well.
5. Reports
5.1. Correlation (affine and splice) table reports
Stratigraphic correlation data is reported in four types of reports (Fig. 5-1, lower left):
- List of affine tables
- Affine tables
- List of splice interval tables
- Splice interval tables
Figure 5-1. Screen capture of correlation reports options within the LIMS Reports framework.
List of affine tables report
This report lists the CCSF alternate depth scales available in LIMS, and the original affine table files that the scales are derived from, one per row (Fig. 5-2). Metadata for each scale are provided as well (see Table 6 for column descriptions). The list can be filtered by entering an expedition number.
To get a list of affine tables report (Fig. 5-2):
- In the Select report panel, expand the Stratigraphic Correlation list and select List of affine tables report.
- In the Hierarchy Search area, filter by Expedition(s), if so desired.
- View data on screen using the View data button.
- Option: download data in CSV format using the Download tabular data button.
Figure 5-2: List of affine tables report.
Table 5-1. Description of List of affine tables report columns.
Column name
Description
Alternate depth scale (CCSF) name
This formal name is auto-generated at the time of upload, based on the contents of the affine table file uploaded by the user and an optional, up to 10 character long extension at the end (after the date) directly entered by the user.
Uploaded affine table file name and link
The file name is exactly the same as that of the original, unaltered user file, which can be downloaded via the link.
Date and time of upload
Date and time based on database server settings.
User comment
User who uploaded the affine table file may have entered an optional comment at the time of file upload, to communicate pertinent process or file information to future users of the data.
Name of account used to upload
Name of the LIMS user account that was used to upload the affine table file.
Affine tables report
This report provides the full affine table information, one core per row (Fig. 5-3; see Table 5-2 for descriptions of columns). Sample identity information (i.e., hole and core) and the cumulative offset for each core resulting from depth shifting, are derived directly from user-uploaded files. However, the CSF-A and CCSF depths as well as differential offset and growth rate are computed in LIMS, based on latest sample registry data for hole and core. The original, unaltered affine table user file can also be downloaded.
To get an affine table report (Fig. 5-3):
- In the Select report panel, expand Stratigraphic Correlation list and select Affine Tables report.
- In the Additional Filters area, use the Alternate depth scale choice list to select the scale corresponding to the appropriate affine.
- You can reduce the choice list by selecting an expedition from the Expedition(s) filter in the Hierarchy Search area.
- View data on screen using the View data button.
- Option: download data in CSV format using the Download tabular data button.
Figure 5-3: Affine table report.
Table 5-2. Description of Affine tables report columns.
Column name
Description
Site, Hole, Core, Type
Sample identity of the core.
Depth CSF-A (m)
Top of core depth from the LIMS sample registry at the original CSF-A depth scale.
Depth CCSF (m)
Top of core depth at the CCSF alternate depth scale.
Depth CCSF (m) = Depth CSF-A (m) + Cumulative offset (m)
Cumulative offset (m)
Distance a core was shifted along the depth axis relative to the original CSF-A depth.
Differential offset (m)
Distance a core was shifted along the depth axis relative to its depth after the next shallower core was shifted. This value, when viewed in context, is useful to detect unusual shifts that may indicate a correlation error or stratigraphic irregularity.
Growth rate
Growth rate = Depth CCSF (m) / Depth CSF-A (m); a measure of the degree of expansion in the recovered sediment section relative to the in-situ section.
Shift type (optional comment)
Recommended terms are: anchor, tie, set, and append. See section 2 of this user guide for explanation.
Data type used (optional comment)
Recommended terms are analysis names or acronyms, such as NGR, MSL, RGB, etc.
Quality comment (optional comment)
Recommended comments are those pointing to problematic shifts based on less than ideal signal.
List of splice interval tables report
This report lists the splice interval tables available in LIMS, one per row, including metadata and links generated by the LIMS or added by the user at the time of file upload (Fig. 5-4; see Table 5-3 for description of columns). The list can be filtered by entering an expedition number.
To get a list of splice interval tables report (Fig. 5-4):
- In the Select report panel, expand Stratigraphic Correlation list and select List of Splice Interval Tables report.
- In the Hierarchy Search area, filter by Expedition(s), if so desired.
- View data on screen using the View data button.
- Option: download data in CSV format using the Download tabular data button.
Figure 5-4: List of splice interval tables report.
Table 5-3. Description of List of splice interval tables report columns.
Column name
Description
Splice name
This formal name is auto-generated at the time of upload, based on the contents of the splice interval table file uploaded by the user and an optional, up to 10 character long extension at the end (after the date) directly entered by the user.
Associated CCSF scale (affine table)
CCSF alternate depth scale (based on affine table), selected by the user who uploaded the splice interval table.
Uploaded splice interval table file name and link
The file name is exactly the same as that of the original, unaltered user file, which can be downloaded via the link.
Uploaded splice tie point table file name and link
Upload of this type of file is optional. File name, if present, is exactly what users named it; link downloads the unaltered file.
Date and time of upload
Data and time based on database server settings.
User comment
User who uploaded the splice interval table file may have entered an optional comment at the time of file upload, to communicate pertinent process or file information to future users of the data.
Name of account used to upload
Name of the LIMS user account that was used to upload the affine table file.
Splice interval tables report
This report provides the full splice interval table information, one interval per row (Fig. 5-5; see Table 5-4 for descriptions of columns). Sample identity information (i.e., hole, core, section, offset) is derived directly from user-uploaded files. However, the CSF-A and CCSF depths are computed in LIMS, based on latest sample registry data for hole and core. The original, unaltered splice interval table user file can also be downloaded.
To get an affine table report (Fig. 5-5):
- In the Select report panel, expand Stratigraphic Correlation list and select Splice table report.
- In the Splice Search area, use the choice list to select the appropriate splice.
- You can reduce the choice list by selecting an expedition from the Expedition(s) filter in the Hierarchy Search area.
- View data on screen using the View data button.
- Option: download data in CSV format using the Download tabular data button.
Figure 5-5: Splice table report.
Table 5-4. Description of Splice interval tables report columns.
Column name
Description
Interval
Sequential numbering of splices intervals added by LIMS.
Site, Hole, Core, Type
Sample identity of the core.
Top section
Section number of top and bottom of splice interval, respectively.
Top offset (cm)
Offset of top of splice interval.
Top depth CSF-A (m)
CSF-A depth of top of splice interval.
Top depth CCSF (m)
CCSF depth of top of splice interval.
Bottom section
Section number of bottom of splice interval.
Bottom offset (cm)
Offset of bottom of splice interval.
Bottom depth CSF-A (m)
CSF-A depth of bottom of splice interval.
Bottom depth CCSF (m)
CCSF depth of bottom of splice interval.
Splice type (optional comment)
Recommended terms are given in Table 4 (Section 2).
Data type used (optional comment)
Recommended terms are analysis names or acronyms, such as NGR, MSL, RGB, etc.
Quality comment (optional comment)
Recommended comments are those pointing to problematic intervals based on less than ideal signal.
5.2. Alternate depths
This report feature provides CCSF alternate depth for any sample and data point in any data report that uses depth (CSF-A) as the primary independent variable. User must select the appropriate alternate scale for a report.
To retrieve CCSF depths with data (Fig. 5-6):
- In the Select report area, select desired data type by expanding the category of interest.
- In the Hierarchy Search area, filter Expedition(s), Site(s), Hole(s), Core(s), etc., as needed.
- In the Additional Filters area, select the appropriate Alternate depth scale from the choice list.
- View data on screen using the View data button. The alternate depths will be shown in the column nest to the CSF-A depths.
- Option: download data in CSV format using the Download tabular data button.
Figure 5-6. Data report example with alternate depths shown in column #10.
5.3. Data by splice
One of the major goals of stratigraphic correlation is to generate “spliced data”, or to retrieve “data by splice”. To get data by splice (Fig. 5-7):
- In the Select report area, select desired data type by expanding the category of interest.
- In the Splice Search area, select the appropriate splice from choice list.
- You can filter the splice list by selecting an expedition in the Hierarchy Search area.
- View data on screen using the View data button. The alternate depths will be shown in the column nest to the CSF-A depths.
- Option: download data in CSV format using the Download tabular data button.
Figure 5-7. Data set extracted for user-selected splice.
Appendix 1: Specifications for affine tables
The affine table exported from Correlator has the 15 columns described in the table below. In order to take advantage of the features offered from within the LIMS database, affine tables exported from Correlator or any other correlation programs must meet import requirements of the SCORS Uploader and Manager application. As a minimum requirement, the first 7 columns must be present.
Affine table column headers
Format
Examples values
LIMS upload file requirements
Upload validation
LIMS report sourcing
Site
text
69, 1103, U1423
Presence of columns required
Validated against identities in LIMS
From upload file
Hole
A, B, C, .....
Core
integer
3, 45, 138
Core type
text
H, X, R, ....
Not validated
From LIMS computations
Core top depth CSF-A (m)
number
2.3, 124.175, 1504.4
Core top depth CCSF (m)
Cumulative offset (m)
-0.560, 4.562, 48.392
Number format validated
From upload file
Differential offset (m)
Presence of columns optional
Not validated
Growth rate
0.91, 1.05, 1.21
Shift type
text
REL, TIE, SET
Data used
NGR
Quality comment
“Interval includes core disturbance”
Reference core
A3, B15, C123
Not reported from LIMS
Reference tie point CSF-A (m)
number
2.3, 124.175, 1504.4
Shift tie point CSF-A (m)
Appendix 2: Specifications for splice interval tables
The splice table exported from Correlator has the 15 columns described in the table below. In order to take advantage of the features offered from within the LIMS database, splice tables exported from Correlator or any other correlation programs must meet import requirements of the SCORS Uploader and Manager application.
Current header in Correlator 2 and 3
Format
Examples
LIMS upload file requirements
LIMS upload validation
LIMS report sourcing
Site
text
69, 987, 1103, U1423
Presence of all columns is required
Validated against identities in LIMS
From upload file
Hole
A, B, C, .....
Core
integer
3, 45, 138
Core Type
text
H, X, R, ....
Top Section
1, 12, CC
Top Offset
number
1.2, 45, 151.7
Top Depth CSF-A
2.3, 124.175, 1504.4
Not validated
From LIMS computations
Top Depth CCSF-A
Bottom Section
text
1, 12, CC
Validated against identities in LIMS
From upload file
Bottom Offset
number
1.2, 45, 151.7
Bottom Depth CSF-A
2.3, 124.175, 1504.4
Not validated
From LIMS computations
Bottom Depth CCSF-A
Splice Type
text
TIE, APPEND
From upload file
Data Used
NGR
Comment
These archived versions and attachments to this page are outdated.
Peter Blum, October 2022
Archive Versions
SCORS LIMS User Guide - 27Sept2022