LMN Solutions Office Tour

LMN Solutions Office Tour

At the LMN Solutions holiday party I was given a Flip UltraHD camera. This is a video of the new LMN Solutions office in the Reston Town Center. I hope you enjoy seeing the place LMN calls home.





Take a look at this blog post for a second opinion.

LMN Solutions - Find out more about us.

LMN Solutions and Social Networking

LMN and Social Networking

A company is only as good as those working for it. We have a great team here at LMN Solutions and it starts at the top. The owners are L M and N. That is Wajeeha Naeem , Matt McKnight . and David Lee . They really care about doing quality work and finding team members with the same values. They always seem ready to talk and collaborate on new ideas. Each of them work hard to keep us happy. On top of this they keep involved with the real day to day work the company focuses on by designing and coding new software systems.

I have not met an LMN employee I would not want to work with. Each of them is an expert in their field but yet humble enough to learn and expand. We learn from each other and have lots of fun completing our missions.

We have found each other through old fashioned social networking. Most of us know each others work from successful work performed at other places.

We are beginning a new phase in the company where we are increasingly finding each other with internet social networking tools. Tools like LinkedIn have been a great help in my career. I have been able to find great people to work with and for using it.

Come get to know our team on LinkedIn, Facebook, Twitter and LMN Solutions Leadership Team.
And come be part of the team.

LMN Solutions - Find out more about us.

Soft Launches

A couple of new things coming out soon from the LMN Solutions Crew.

Be on the look out for WhatsFunThisWeek  - which is a nice little site we built for a new client to track upcoming local events and keep track of fun lists.  It should be launching in December.


Wealso plan on moving a lot of the essay type commentary around particular topics over to our new Agile Blueprints site. Right now, there is nothing there. Soon, it will be a font of knowledge. So, extremely soft launch on that one. By launching something so small, every small improvement we make will seem large.


We're also working hard on a new application in the employment space. This has been around the bend a few times, but we have the right product development team putting it together and will have two interesting products of this effort emerging soon.

In terms of new services, we are making a lot of progress in delivering geographic business analytics as a service. Using ESRI Business Analyst we are delivering geocoding and demographic analysis services.  We are using these tools and data to combine what we know about site selection algorithms with commercial problem sets. 

Finally, we're developing a high concept product in the text processing and analytics space. We don't have a clear launch for this yet, we're still very early in the customer discovery process trying to figure out who might want it and how they might want it.

In all of these cases, we're trying to lower the barrier between what Merlin calls "blue sky solutioneering" and getting a Minimum Viable Product out the door to test the waters and validate our concept of what the user wants.  More to come.

Palantir & Human Effort

Palantir & Human Effort

Palantir Government is a product we have been working with for some time here at LMN Solutions. Overall it is a great product full of features to aid in investigating events and people. Because of its power and many features it can be hard to learn and applied to situations where it does not fit.

Palantir Government has been used to investigate a cyber attack and many other events. It has an visually stimulating interface. It is centered around three types of objects - documents, entities and events. Each of these are sub divided into other groupings like meetings, bombings, cleric, government officials and so on. Each of these objects can have hundreds of properties such as name, address, longitude, latitude, date, birth date and any other type of data you can think of. It is easy to get lost in the details.

To make it possible there is a good amount of human effort involved to really make Palantir shine. The people using it and the IT folks supporting and loading it with data need to focus on learning how to make it work best for their particular group. Once you get the right data into it and then learn how to filter and search through data the linkages it can build are amazing. It is a "connect the dots" technology.

But it requires patience and time to iterate through the learning process to get the data into the right form and loaded into the system. It takes an experienced staff to really think deep about the ontology needed and how the goal of your group fits into it. It takes an experienced administrators to manage the Oracle database and the other Palantir servers. Experienced personnel to convert data from other systems into rich objects that Palantir loves can also aid the process.

Sometimes it is necessary to remove data from the system or change the object structure or ontology to help the end user, the analyst, work faster. Sometimes too much data can muddy the picture and perhaps slow the system. It takes strong communication with the analyst to figure out what needs to be done. We recommend a collaborative effort. Get the IT folks supporting Palantir right down to the analysts desk.

One of the keys to a successful implementation of Palantir Government is to define the goal. You must not just start ingesting the entire contents of all of the data repositories you have access to. It takes a lot of thought and preparation. Then do not give up, keep trying to add focused data sets that will strengthen the links between events, people and documents. Much of it comes down to old fashioned investigation and slowly building the data to link things together. At times it can just be boring data entry and tagging of documents. At least Palantir makes taging of documents easy.

Palantir Government makes finding linkages between people, places, events and documents easier but investigative work is never easy. It is vital to put in the time to learn the tools that will help find what you are looking for. Palantir has a excellent free training game called Operation Tradestop. It really helps you learn how the search tools work on data from actual events and people.

Palantir supports the use of custom helpers written in Java. They can be deployed right into the Palantir desktop. These are mini applications within Palantir that add additional functionality. With the use of helpers the tasks of analysts can be focused right to the tools they need without having to worry about all of the other features Palantir offers.

LMN Solutions has built a series of Palantir Government Helpers that aid in focusing the tasks of data entry and making sure the analyst does not get lost in a sea of properties.

• Jabber Helper - real time communication within the Palantir window. Send Palantir objects straight to a colleagues Palantir graph. Collaborate with colleagues that do not have Palantir
  


• Tasking Helper with work flow coordination. Assign Palantir documents to others and manage tasking.


• Link Builder Tool - drag and drop interface to quickly build linkages between objects.
  

Overall Palantir Government is a useful tool, just be ready to put in the effort to narrow the focus of your task and spend the time needed to fully understand and learn the product. Palantir is not a panacea. One cannot just force all of the data and documents into in and expect it to magically determine the linkages between people, events, places and documents. If you can put the necessary time to see it through Palantir will yield results.

Come visit our booth at the Palantir Government Conference GovCon4 October 13, 2009

LMN Solutions - Find out more about us. Need help with Palantir, contact us.

The Value of Estimating (Even With Software Projects)

There is a trend in software process commentary to claim that plans and estimates aren't very useful, to admit that most of the plans we make are, in effect, guesses. There is some truth to this. In many cases, the scope of the work is unclear, the staff available to complete it is unclear, and the difficulty of creating the application is unclear.  On the other side of the fence, there are fixed budgets that have to be requested, there is the PMBOK which encourages you to build a Gantt chart including every task on the project with no uncertainty, and there are a lot of people out there that want to describe the software development process as something that is more like a manufacturing or construction project.  In reality, software development projects lack the predictability of those sorts of projects, because they involve invention of something new.

However, it is absurd to say that estimation is impossible. For example, it is fairly easy to give an order of magnitude size estimate of a project based on other similar projects. Even if your own organization is not keeping track of it's historical project performance, industry standard tools such as Construx Estimate can be used to provide some rough guidelines based on their collection of data.  These are overall project estimates, which can be superior to the bottom up estimation that is often done in agile projects.  The estimation process by which each feature or story is individually estimated can problematically leave out costs and time associated with missed requirements, tasks that are not particular to a feature, and the overall drift of iterative processes by which the initial implementation of a feature is not final, but must be refined over time with a refactoring cost that is often not estimated.  Whole project estimation techniques avoid some of these oversights, and can even be tuned to avoid the error in the initial size estimate for the system.

It is also possible to give estimates that demonstrate the uncertainty of the project by including a probability curve or range. The estimate also should not be static, but rather updated as new data comes in.  The Cone of Uncertainty is one common way to represent this process:

 

This particular manifestation of the cone represents a waterfall style process underlying the estimation accuracy, but we can imagine a similar cone being applied to agile methods. When work is done to begin the bottom up estimation process, there is more information available now to map that information to the estimate, and eventually we can switch over to the bottom up estimate when it starts to become more stable than the holistic estimate.

The key thing to remember here is that an estimate is not a commitment.  If you commit to a the 50% confidence estimate, there is a 50% chance you will miss the date.   If you commit to a 90% estimate, the proposed timeline may be unacceptable to your customer, especially if you are willing to admit that there is some failure rate involved in software projects and there is a non-zero probability of not delivering at all.  Making a commitment to a 90% date also introduces the risk of Parkinson's Law coming into play. When someone writes out a Gantt chart that has the 90% confidence date for each individual task, seemingly magically, the work comes in very close to the deadline. Except for those few tasks that don't, the ones that weren't actually started in earnest until the deadline was right upon the developer, but were more complex than expected.  If those were on the critical path, all of sudden your whole project just got pushed out past the 90% confidence date, even if everything else is completed by the deadline.  This where Critical Chain Project Management comes into play, but we'll save that topic for a future post.

Selenium Remote Control

Selenium Remote Control - Web Interface Testing


When you watch this you might want to turn on closed captioning as the volume is very low. To see the actual text full screen mode is best.

Test Automation

Selenium Remote Control has been an extremely helpful tool in increasing the velocity of our Agile Scrum team by shortening the time between development and testing.

It is a tool that actually manipulates the browser like a real person would. This allows the javascript as well as the server side code and database to be tested as a whole.

In conjunction with various Agile techniques the process of building automated GUI tests has opened communication between testers, developers, customers and management. Surprisingly it has been useful as a sort of management dashboard to give high level managers the assurance that development and testing is progressing. People seem to enjoy watching the application run automatically and seeing green and red bars pop onto a report in their web browser.

With some integration with other products it seems to wean folks off of those tractability matrix reports also. Using ant, jUnit reports and annotations has allowed a loose mapping of a story to actual test results.

LMN Solutions has experience in integrating tools to enable continuous integration, automated testing, unit testing to support the Agile process. This demo was built using Eclipse, jUnit, Selenium Remote Control and Selenium IDE. The application under test, LMN Jobs and LMN People was built with Ruby on Rails and jQuery.

On other projects we have combined tools such as ant, Cruise Control, jUnit, Fitnesse, watir, Ruby on Rails and Java to enable continuous integration with functional testing and reporting.

Orthorectification

Orthorectification

Update: Co-authored paper published by Journal of Applied Remote Sensing: Automated imagery orthorectification pilot


Photo left: Raw Film image from USGS archives.

Orthorectification, just a fancy word for processing images taken from the sky to make them geographicaly correct.

If you have used Google Maps or Google Earth you are a consumer of orthorectified images. Or imagery as the industry calls it. If you ever notice an image of the ground that looks sort of distorted, flat or appears to be taken from an angle it could be a image that has not been orthorectified yet.

To simplify it the output of perfect orthorectification should be an image in which you can point to each pixel and determine the true geo coordinate and elevation for that pixel. It tries to generate an image in which the ground of the image appears to be viewed from directly overhead (commonly called nadir). Of course anything sticking up above the ground will typically not appear correctly without a huge amount of additional processing.

The purpose of doing all of the work necessary is to increase accuracy. When all of the imagery is accurate it is possible to then mosaic it all together so there are no seams between images. It is also possible to take imagery taken from different time periods and layer it on top of each other to watch for changes. It enables all sorts of time saving tasks in many industries such as oil, geological work, scientific, electrical transmission, military, civil engineering and more. In the EPA it has been used to discover where pollution is buried by comparing historical images with each other to detect change over time.

Simplified Process Overview

The most accurate form of orthorectification involves sending surveyors in to the field to collect data points called Ground Control Points or GCP. Using GPS units they collect the longitude, latitude and elevation of points on the ground that are visible from overhead images. These are points such as road intersections or visible land marks. The problem is the cost of this option.

Many resort to the next best solution which is to use existing orthorectified images from the past. These are called control imagery.

The next step involves finding the ground control points from the survey or control images and matching them to the same landmarks in the raw images. This used to be a manual process. Someone had to locate these points and then correlate them together. There is now software that can find these points automatically and one can just accept what the software determines or perform a manual review of the points collected.

One must collect 3 GCP per image otherwise the final warping process will produce some strange results. If you ever took trigonometry 3 points are needed to determine the three dimensions of space. With only 2 points one cannot determine the height the photo was taken from and the scale of the photo will be wrong. Most tools require about 8 or so points to be collected and they should be spread throughout the image.

Two tools I have used for automatic GCP collection are PCI Geomatica and Erdas Imagine. I have also come across a module for ossim that does this also but I have not tested it yet. Other tools will perform orthorectification but require you to collect GCP manually.

It appears that PCI Geomatica collects these points by analyzing the high contrast areas where roads cross or the edges of landmarks.
Example of DEM from NASA.

The next part involves a file that contains the elevation model for the area where the image was taken. Often it is a TIFF file that contains pixels of varied shades of grey. Each pixel represents an elevation value by its color and has a geo-coordinate assigned to it. This allows the software to process the image correctly. This is called a Digital Elevation Model or DEM. If the elevation model is not included the process is sometimes called rectification and not orthorectification.

The next stage involves computing a math model called Rational
Polynominal Coefficients (RPCs) using the control image, the digital elevation model, raw image and the ground control points. Many times the camera model is included. This gives the software the distortion values that a camera lens causes.

Now the software takes this math model and starts warping the image. It moves pixels around to make the image as correct as possible. In areas with lots of elevation changes like mountain ranges the finished product will eliminate areas where optical illusions will make a hill look like a depression.

It has been proven that this entire process can be automated but with less accuracy than with an experienced operator. One of the issues is that each sensor or camera source has to be optimized. Each camera or satellite sensor has its own characteristics and the software has to be re-configured to optimize the results. Processing Ikonos images is going to be different from QuickBird. Aerial imagery from aircraft is even more complicated and has less standardization. This is not really a problem if someone has a large amount of imagery from a small handful of sensors.

To see a visual example of orthorectification view this image.

In a future post I can describe the details of the process used on historical aerial imagery.

Resources

http://www.satimagingcorp.com/svc/orthorectification.html
PCI Geomatica
ERDAS
OSSIM - Open Source
Automated imagery orthorectification pilot

LMN Solutions - Find out more about us.

Automated imagery orthorectification pilot

J. Appl. Remote Sens. 3, 33552 (2009)

http://link.aip.org/link/?JARSC4/3/33552/1