EXP390 Application Developer Report

Contributors

  • Tim Blaisdell

  • Dan Cary

Summary

This document highlights changes to the JOIDES Resolution laboratory data management environment during Expedition 390.


Selected issues are highlighted, but not reviewed in exhaustive detail. In general, see the ship activity log and product pages on the developer site for additional detail: {+}http://banff.iodp.tamu.edu/display/DEV/Developer+Home+Page+
The activity log specific to this expedition will be placed here: {+}http://banff.iodp.tamu.edu/display/DEV/Ship+Activity+Log+

Special Projects

During Expedition 390 the developers worked on the following special projects, outside normal expedition-related duties:

  • MUT - Worked on converting MUT to a modular design where each analysis is a separately-deployable module, retrieved and loaded dynamically at run time via LDAQ's resource manager.

    • Modules are built as .zip files, dropped in the LDAQ_PRODUCTS folder, and deployed like LDAQ modules to /ldaq/resources on the server.

    • All analyses have been modularized, but not all have been well tested.

    • This new MUT is deployed as "MegaUploadaTron2".  Both "MegaUploadaTron" and "MegaUploadaTron2" can be installed on a computer, but both cannot be run at the same time.

    • See the Change Summary section for more about this.

  • AUTHER - Worked on completing the new rewrite of the IODP authorization tool.

    • Shore based project facilitating Role/Account/Privilege management for all IODP applications.

    • Tasks involve user restrictions to features based on developer status and role membership. 

    • Changes were demonstrated to shore personnel and deployed to test servers. 

  • SDRM - Shore-based replacement for the old SaDR applicaiton.

General Duties Performed

During Expedition 390 the developer participated in:

  • Routine expedition support.

  • Maintenance of software applications (as detailed in the sections below).

  • Assist with data management in cases where LIME and other software tools do not suffice.

  • Other duties as assigned.

Change Summary

 

Product

Notes

MADMax

Several extensive changes were made that were subsequently backed out,  These changes had to do with giving the user the ability to perform multiple measurements on a sample, and then select which resulting values are displayed in MADMax and used in MAD_Calc calculations.

These changes have all been backed out and the current head revision in SVN does not include them.  

To get back the changes, check out MADMax at revision 8796.

In addition to the above, the following changes were made to MADMax (which are still in the production version):

  • When a measurement is completed, and MADMax sees that all measurements required for the current MAD_Calc method have been completed, MADMax immediately pops up a confirmation dialog asking if the user wants to perform MAD_Calc.  This is to prevent the situation where a tech or scientist has forgotten to run MAD_Calc for several days and someone has to go through the rows one by one and run MAD_Calc for each.

  • The LabVIEW modules were re-built using LabVIEW 2201.

SampleMaster

A defect was discovered during the expedition that enabled the SampleMaster user, by one fairly simple action, to bring the Tomcat servers to a grinding halt.  The defect was fixed, but its nature is good to know.

In the hierarchy browser, select QAQC -> CORELAB -> OTHER, then go to the Edit tab.

Doing this would cause SampleMaster to try to retrieve all child samples of this parent, along with all tests and results for these children.  This amounted to many millions of records that the server would try to load, convert to JSON, and deliver via HTTP, which might take several minutes and would consume 100% of CPU and memory on the server.  

On top of this, due to the delay at the client, the user might click around in the hierarchy browser controls, causing the above request to be reduplicated several times.

This was fixed by changing the code to load only what analyses it needs to fill in its tables, rather than all analyses.

A couple other very minor bug fixes were made.

MegaUploadaTron
a.k.a. MUT

  • As mentioned above, a lot of work went into modularizing MUT so that each analysis is represented by a separate, independently-deployable module that gets loaded at run-time.

    Since we were not able to fully test this change to our satisfaction during this expedition, two versions of MUT now exist and are deployed to Olympus and are installable via IODPLauncher: MegaUploadaTron and MegaUploadaTron2 (referred to as MUT2).  Eventually, when we are confident that MUT2 works at least as well as MUT, we want to drop the "2" from MUT2's name and deploy MUT2 as just plain "MegaUploadaTron".  At that point, there will only be one MUT, which will be used everywhere, as before.

    Currently, MUT2 is deployed and fairly well tested on the following workstations:

  •  

    • SHIL

    • MFK2 spinner

    • STMSL

    • Alkalinity


  • Some changes to the analysis code, particularly WRLS and KAPPA, were only made in the MUT2 module, so if you revert to the old MUT on those workstations, you will not be getting those changes.


  • The WRLS upload was changed to fix the bug where it set x_display values improperly.  Again, this change is only in MUT2's "MUT_WRLS" module.

LIVE

Changed so that, when building URLs for thumbnail images, it uses TEST.X_PRODUCT rather than SAMPLE.X_EXPEDITION for the catalog value.  This was to fix an issue where LIVE could not see images uploaded on the current expedition, for samples brought out from previous expeditions.  

In the usual case aboard ship, TEST.X_PROJECT is the same as SAMPLE.X_EXPEDITION, so it doesn't matter which is used, but when samples from previous expeditions are brought out for further testing during an expedition, SAMPLE.X_EXPEDITION will be the previous expedition, but TEST.X_PROJECT will be the current expedition, and that's where ASMAN is storing the new images.  Therefore, LIVE should always use TEST.X_PROJECT.

LORE

Not a code change, but uppdated many records in NAV_HIERARCHY so that the "Acquired after" search filter would be enabled for all reports.

ThermCon

During the expedition, a scientist at the ThermCon station tried to distinguish one measurement from another by adding some text to the folder name where the software stores the data.  This caused the upload to fail, because ThermCon expected the folder name to be in a precise format.  

ThermCon was changed so that it would be more forgiving about the folder name, allowing additional text after the value it looked for without failing the upload.

limsW

A change was made to the ResultLog service so that a result can be logged with no value.

This was to fix a problem in CahnBalance where it would not upload any results if the container number was not filled in.  However, it was determined and generally agreed that it ought to be possible to log a result with an empty value.

ImageGrabber

A refresh button was added to the UI so that the user does not have to exit and re-launch to get a new site that was just added.  

ImageCapture

Strip images of orientation (landscape vs. portrait) information before upload.  This was done because the motion of the ship tricks the camera into thinking it's being flipped this way and that, so that it randomly writes image files with bogus orientation information.

Catwalk

A minor change was made to some auto-fill formulas so that values for volume in a template would not be overwritten by the auto-fill formula.

Outstanding Issues from EXP392

  • ImageCapture is sometimes throwing an error when running a function in the Atalasoft code.    This was reported on the last expedition, but never happened on this expedition and we did not investigate it at all.

Development Tooling and Infrastructure Changes

  • Throughout most of the expedition, we did not have access to several shore resources, such as shore databases (via SQL Developer), shore Tomcat servers and the shore Nexus server.  All of these issues were resolved during the expedition by MCSs working with shore IT personnel.

Hardware and Server

No hardware or server changes were implemented by developers during this expedition.

Â