Chesapeake Times
Technical Review, Issue 3
October 2020
|
|
Letter from the General Manager
Over the past few months, I was able to get proficient at Zoom. It wasn’t on my to do list in January. Not a perfect replacement for the old ways, and I am sure, like many, the travels and in-person meetings are missed. Working with our clients is essential, and we will do whatever it takes to make it happen. Helping a client with their data, teaching a class to new users, or working with our partners on new development, this mode seemed to work out. I was taught that you never stop learning, and it is true; I've
learned a few lessons using this new technology. Maybe the most important one is don’t eat lunch on the call. I can just imagine someone on the other end had a large screen while watching.
At Chesapeake, we are putting our final touches (ok, let’s call them bug fixes) on our next release, SonarWiz 7.7. This version has over 50 updates – some small, some big, and hopefully everyone finds them useful. We also took note of what to do next – and are building on a plan for post 7.7, as we continue to add user requests, making the program easy to use, efficient and innovative.
Our challenge has always been to keep up with the industry changes – new systems and formats, new processing techniques, and new standards required by clients. We also deal with larger data sets and higher resolution from these systems. Making a mosaic at 3 cm for a survey area. Processing 2GB data file. Years ago, these were things we could never imagine doing because of computer and software limitations. Now we can. It is hard to imagine the next 10 years of advancements – and challenges we then need to meet. We are laying the foundation now for handling these requests that will be asked of SonarWiz. So, you may not see or use the 50 updates in SonarWiz 7.7 today. But you are likely to use them at some point in the future.
Keep well and stay safe!
- Harold Orlinsky, General Manager
|
|
SonarWiz 7.7 New Features and Technical Notes
Cloud Considerations for Survey Processing .............. Jonathan Fleetwood, Engineer
Developing Specifications for SonarWiz on AWS ....... David Finlayson, Chief Scientist
New Sub Bottom Reflector Digitizing Tools ................ Christopher Favreau, CTO
Large File Handling .................................................... Harold Orlinsky, General Mgr
Reconnecting Project File Path References ............... Patrick Zynda, Support Engr
SonarWiz Annual Training .......................................... Stephen D'Andrea, Bus. Mgr.
|
|
Recently we've seen a good deal of interest in cloud platform services and related inquiries into how SonarWiz would run in their environments. Companies stand to see a lot of benefits from these relatively low-cost, scalable, externally-managed resources, and we've been investigating the various opportunities and challenges involved in utilizing them.
Originating mainly as internal services used by large tech companies, cloud platforms started to be offered to the public in the early 2000s. Initially utilized by customers primarily as hosting and web software delivery platforms, cloud services have seen increasing use as hosts for traditional desktop software since then. Their availability from any networked location and flexibility in machine configuration are beneficial
to many kinds of software and related workflows.
Cloud instances can be tailored extensively - many options are offered, allowing companies to choose machines that are focused toward most reasonable blends of CPU and GPU capability, memory, disk size and speed, and network bandwidth. Allowing for a mix of machine configurations in the pipeline can help with maintaining a good cost to performance ratio, especially for large projects.
Early in a project, only moderate computing resources are necessary in planning out lines to cover a given survey area and estimating hours required to achieve a desired coverage. A fast network connection can help if using pannable online maps to orient within the survey area, but otherwise, a low-powered machine will suffice.
After collection and upload of raw files to storage, higher disk bandwidth is useful for import; all of navigation, bottom tracking, bathymetry merging and gain processing can benefit from this as well, with the addition of strong CPU capability.
Interpretation benefits from moderate-to-strong disk capability, and may benefit from GPU in some instances. For deliverable creation and export, disk throughput and latency come to the forefront again, with strong CPU helping somewhat, just as with pre-interpretation processing.
After delivery and acceptance, project data can be archived; cloud platforms offer low-cost cold storage options for infrequently-accessed data. The data won't be available to work with quickly when stored in this manner, but the cost savings over faster storage is substantial.
Workflows with "spiky" resource needs over a weeks- or months-long timescale can realize a related benefit to the configurability of cloud platforms due to their on-demand nature. Instances with moderate resources can be maintained for the lower baseline activity level - line planning, contact picking, and so forth, with heavier instances brought to bear during more intensive processes such as import, gain application, and
geoimage export.
In some instances, cloud platforms can also offer increased options for collaboration. The model of centralized storage coupled with distributed access offers the potential for different numbers of people to work on a project at different times, depending on need. Currently SonarWiz stores the project as a folder on a disk rather than as, say, centralized in a concurrently-accessible database, making it unfeasible to have multiple people working on exactly the same project.
That said, splitting a survey by geographical area, sonar type or frequency, time or vessel can be used to achieve many of the same benefits. Splitting projects in this way can be done with centralized on-premises storage, but that comes at the cost of maintaining the infrastructure required to allow for off-site access, something that is built into the cloud model.
We are excited about the possibilities offered by cloud services, and are making steady progress toward ensuring that SonarWiz and related tools work well in these environments. There are some challenges to overcome, but the benefits that can be realized are worth some investment.
- Jonathan Fleetwood, Engineer
|
|
Developing Specifications for SonarWiz on AWS |
|
SonarWiz 7.7 will be the first release of SonarWiz to be officially supported on Amazon AWS. As we prepare to release this new version of the program, we have been working to develop best practices and recommendations for setting up an AWS environment that will run SonarWiz as efficiently as possible. We developed a short list of typical sidescan operations that we are using to benchmark different AWS configurations.
In our initial experiments we tested 13 different AWS configurations and one laptop for comparison and measured how long they took to complete our benchmark tasks in SonarWiz. The results are shown in Figure 1.
|
|
Figure 1: SonarWiz benchmark results for each AWS instance. Lower times represent better performance. The Lenovo T470p laptop result is indicated in red.
There is both good news and bad news in these results. The bad news is, that even monster machines like the winning g4dn.16xlarge with 64 vCPUs and 256 GiB of RAM only achieved about 1.8x the performance running SonarWiz that we can get from a ThinkPad laptop. This has to do with a number of performance problems that occur when running desktop applications like SonarWiz on cloud platforms that were designed to share resources across web servers.
The good news is that you do not need to rent the most expensive machines available on AWS to achieve near-best performance. This latter point can be better seen if we plot benchmark performance versus the rental cost of the instance as in Figure 2.
|
|
Figure 2 Cost per hour to rent the instance versus total time to run the benchmark.
Machines in the lower-left corner of Figure 2 offer good performance running SonarWiz and low rental cost. In fact, instance g4dn.4xlarge achieved a benchmark time only 1.04x slower than the winner but cost only $1.94 per hour to rent and you can get pretty good performance out of machines that only cost 50 cents per hour. There is no reason to rent big iron if SonarWiz cannot yet fully take advantage of the additional resources provided by these huge servers. We found the sweet spot was a GPU-equipped machine with 16 to 32 vCPUs and 64 GiB of RAM.
All but three machines we tested were faster than the laptop we include for reference, but not by more than 2x. More than performance, the real win of running SonarWiz in the cloud is the flexibility it provides. Once you have created an image with SonarWiz and loaded your other mapping software, it is easy to log into that image from anywhere in the world, even from computers with vastly inferior power, like a Chromebook or the lab computer that hasn’t been replaced in years.
Cloud computing is the future of the industry and with the release of Version 7.7, SonarWiz will start that journey.
-David Finlayson, Chief Scientist
|
|
New Sub Bottom Reflector Digitizing Tools
We have some new features in SonarWiz 7.7 that will assist in digitizing reflectors. The first tool is the Smart Reflector Tracking tool. It assists the user in tracing part or all of a reflector. It turns a job that used to entail many clicks into 2 clicks. Just enable the Smart Reflector Tracking tool at any time when in the Sub Bottom Digitizer Acoustic Reflector Tools by clicking the Auto-Fill button.
|
|
Then the user simply sets a starting “anchor” point and an ending point. The tool then traces the reflector automatically between the 2 points. The user then can continue tracing by setting another ending point or by starting a new reflector. |
|
Set the anchor point, scroll to the end, set the end point |
|
The start anchor point does not need to be visible in order to do assisted tracing. The user can set an anchor point at the start of a file, scroll to the end and set the end point. The auto-fill will trace the entire reflector between the start and end points. |
|
Zoom out to see the whole automatically traced reflector
|
|
The Smart Reflector Tracker can also assist in cleaning up manually digitized reflectors. The user can enable it whenever they would like during creating a new reflector or editing an existing digitized reflector. In the event that the Smart Reflector Tracker has a difficult time the digitized points can be manually edited. |
|
The next tool we are introducing in SonarWiz 7.7 is the multiple reflector identification. This tool assists the user in visually identifying reflectors and their harmonics. After the user enables this tool they just move the mouse cursor to the reflector in question and then compare the harmonic markers with the actual data. This is done interactively. If a harmonic is identified then it can be marked temporarily by clicking. |
|
Multiple reflector identification tool in action |
|
We will be extending this capability in the future to further assist the user in digitizing reflectors.
-Christopher Favreau, Chief Technology Officer
|
|
Large File Handling
In the next version of SonarWiz, we added a few options to help folks with large files. We have seen the ever-increasing file sizes, as folks collect higher resolution data or just more data. Days of 8-bit sidescan and single beam echosounder have been replaced with 24-bit sidescan data and multibeam sonar. With each update, file sizes are larger, and sometimes, unmanageable to move from one machine to another, or take much, much longer to process. We have added in a few new features in the latest update to address this issue.
|
|
Tool to split XTF files by sensor type
An XTF file is a catch all, a common type format that the industry has been using for a while. And because of the format as a wrapper, any sensor type can be made to an XTF file. Data loggers can create a single XTF file with multiple sensors, often large files, which can make processing slow as the entire file needs to be read. This feature can be found in Tools > XTF.
|
|
A single 120MB XTF file, for example, having both Sidescan and Sub bottom, would get split into two separate files, each appended with a file name that indicates which sensor type is included. Included in the split file is the ancillary data needed for complete processing of that sonar type. |
|
In my time trials on a laptop, I was able to load these split files 25% - 30% faster than the single XTF file with both sensors.
Another tool added in the latest release is to split bathymetry data files – the CDF files created by SonarWiz. Four different options exist to split these data files – visually using a mouse, by time, ping, or feature.
|
|
Creating the data files to be bounded inside an area allows one to segment the data for applying specific sound velocity data, removing unwanted turn data, or splitting data files for processing in a specific region. |
|
Original bathy line, imported /shown in SonarWiz |
|
Split file, removing both western & southern section |
|
This tool will come in handy, letting the user split data files of any sonar type and creating a smaller file for data processing. With the smaller CDF file, data processing time is reduced.
-Harold Orlinsky, General Manger
|
|
Reconnecting Project File Path References in SonarWiz
The structure and nature of SonarWiz projects require various connections from items created and images or databases in your project folder. Our support team frequently hears from customers saying they are missing contacts, files etc. especially after a project has been moved from one workstation to another. To address this common issue, our developers created a tool to reconnect these file paths. It is located under the Tools tab in the program, Locate Project Files…
|
|
The goal here is to see all green status and repair path names that have become broken due to a project move or other file operation. SonarWiz stores path names in 5 different databases; some of these databases have several paths that all need to be accurate for the SonarWiz project to operate correctly.
For example, we often see issues with sonar files being lost in projects after moves, so let's walk through reconnecting those items. What happens when you get this error message from SonarWiz?
|
|
Our support team recommends starting the troubleshooting by opening your project .MML file in a text editor like Notepad which is the index SonarWiz uses to look for files that should be in the project CSF folder. Use CTRL + F to find the following line: <m_nNumFilesInProject>5</m_nNumFilesInProject>
When you open a SonarWiz project, the program looks at this line and attempts to match the items in this list to the contents of the CSF folder. If you received the error message above or there are files missing when you loaded the project, one of these lists (.MML or CSF folder) didn’t have the file. It is often the case that these CSF files were left behind on a network drive or another workstation. So you can use the Add CSF Files… command in the File Manager to bring them back into the project.
|
|
For sonar files, you will need to make sure all the extension files created in conjunction with the .CSF files are in the same folder when bringing them back into the project. They will all have the same name with varying extensions (.CDF, .BTK, .CSD) and contain the necessary settings.
Our support team wrote an article in our Knowledge Database in Confluence (linked here) with a complete list of items that need to be reconnected in the Locate Project Files tool that users can reference. If you have any issues when attempting to reconnect path/ file references, our support team can assist further.
- Patrick Zynda, Support Engineer
|
|
SonarWiz Annual Training
This year Chesapeake Technology will be hosting our Annual Training event virtually, allowing easy participation from the comfort and safety of your home office! While the environment will be different, the syllabus will be just as in-depth and interactive.
Classes will be held December 15-17, 2020 from 10am to 6pm EST. Attendees will learn the latest features, best practices and tips to improve workflow and efficiency using SonarWiz to process data and survey plan. We are pleased to welcome our hardware partners and clients for guest speaking segments throughout the event.
The cost is $249 per person or $695 per company for up to 4 participants. Registrants will be provided with a two week license of SonarWiz 7, sample data sets, training materials and other guides. Click below to register, call +1 650 967 2045 or email us at workshop@chesapeaketech.com if you have any questions. Hope you can join us!
-Stephen D'Andrea, Ashley Chan
|
|
Privacy Notice: Your email address (and other personal information, if applicable) is being used to fulfill your request to receive newsletters from CTI. Use is subject to CTI’s Website Privacy Policy. CTI complies with the EU's GDPR laws. Continued reading of the newsletter is implicit acceptance of our Policy, and you may unsubscribe at any time using the link below.
|
|
|
|
|
|
|
|
|
