Friday, May 28, 2010

Keywords Management in Statement Analyzer

Mark McClish's Statement Analyzer is missing an important feature to manage keywords and descriptions. There are many reasons to need more keywords, whether some are missing or simply because one's field of expertise is different from law enforcement or does not need to perform statement analysis on incident reports. This article will demonstrate how it is possible to manage keywords and descriptions.


Statement Analyzer and OpenOffice Calc are necessary to manage keywords and descriptions. OpenOffice Base could be useful but optional and not presented in this article.

Statement Analyzer Databases
There are seven (7) databases used by Statement Analyzer and they are located in C:\STMTANAL\Data\. These databases are xBase/dBase compatible since it is the native database format of Visual FoxPro. Further information can be read in xBase file format and its data types.
WARNING Modifying databases could prevent further usage of Statement Analyzer. Backup everything in the directory C:\STMTANAL\Data\ before opening or modifying any database.
ANALDESC.DBF (RECNO,N,5,0; DESC,M)
This database contains descriptions to be associated with their respective keywords. The data record is defined by a record index identifying the record and a description.
ANALKEYW.DBF (KEYW,C,50; RECNO,N,5,0; GROUP,N,3,0)
This database contains keywords associated with their descriptions and respective groups. The data record is defined by a keyword, a record reference to a description, and a group identification number.
ANALMAST.DBF (ANALNAME,C,25; ANALYSIS,M)
This database contains saved statements for later retrieval in the software and it is referred to Analysis Master File. The data record is defined by a saved statement name, a.k.a. filename, and its contents. The first valid record starts on the third row. This tutorial does not use any data in this database.
ANALWORK.DBF (KEYW,C,50; COUNT,N,4,0; RECNO,N,5,0; GROUP,N,3,0)
This database contains data used while a statement analysis is performed and it is temporary only data. This tutorial does not use any data in this database.
CCSLOG.DBF (DATE,D; TIME,C,8; NOTE,C,45)
This database contains logged user actions and other debugging information. This tutorial does not use any data in this database.
CCSPARAM.DBF (COMPANY,C,50; COMPADDR,C,50; COMPADDR2,C,30; COMPCTYST,C,30; COMPZIP,C,10; SUPERPASS,C,24; USEPASSW,L; EPOCH,N,2,0)
This database contains data used to define system parameters. This tutorial does not use any data in this database.
CONFIG.DBF (TODAY,D)
This database seems to be unused and this tutorial does not use any data in it.
English Transitional Words
Statement Analyzer provides an extensive list of keywords to be used during an automated statement analysis. Unfortunately, the list is incomplete and some important keywords are missing. English Transitional Words, often labelled as text bridges in the law enforcement community, are meaningful in statement analysis.

There are literally hundreds of transitional words and I've learned many of them. A good place to obtain an exhaustive list is the English Transitional Words Database 2.0 website. They list more than 1300 transitional words though some can be found in more than one category.

FBI articles referred in Detecting Deception in an Apology list as little as six (6) different categories in contrast to the 42 available on the website mentioned above. These six (6) categories contains around 50 unique transitional words and are good enough for a quick screening on incident reports but more transitional words are necessary to perform a thorough analysis. For example, transitional words that have a grammatical function of time will help a law enforcement officer to quickly and properly lay down a timeline in a particular incident and look out for gaps — which shall be further investigated.

Performing statement analysis on different statements than incident reports, such as an apology, a comment, an email, and so forth will also require to understand more transitional words.

Adding Keywords
Let's assume one desires to add keywords that are transitional words with a grammatical function of contrast because they are currently not available in Statement Analyzer. The following contains a partial list of words with a grammatical function of contrast, see English Transitional Words Database for more.
however, in contrast, indeed, instead, nevertheless, on the contrary, on the other hand, yet.
OpenOffice Calc offers an excellent support for dBase databases. Make sure Statement Analyzer is not running and open ANALKEYW.DBF using OpenOffice Calc. OpenOffice Calc will attempt to import the database and a window may prompt a choice of character set. Statement Analyzer uses ISO-8859-1, choose “Western Europe (ISO-8859-1)” and click “OK”.

The first sheet contains every keyword Statement Analyzer has. There are three (3) columns available as previously explained. Move to the first available row, which should be around cell A765. Enter the list above with a keyword, 102 and 0 per row.

The last known description record index is 101 and it is necessary to use 102 to create an next description for these transitional words. For an inexplicable reason, the word “ground” is duplicated (see row 760 and 762) one with a record reference of 100 and group 156 and the other with a record reference of 101 and group 154. The latter is a mistake and could be deleted, whereas we would use a record reference of 101 and upward for any additional keywords.

The group is 0 to denote “no group”. Grouping is used to denote change in language, e.g., when a person use the word “car” throughout a statement and later use the word “vehicle” when describing the actual crime, and is an indication of deception.

Both “car” and “vehicle” are part of the group number 147.


Save once the keywords are added. OpenOffice may prompt a choice as whether to keep the dBase file format or convert to ODF file format. Choose “Keep Current Format” to maintain compatibility with Statement Analyzer. The keywords will not be available as long as the record reference to a description is not available, see next section.

TIP Reindex the keyword database in Statement Analyzer upon any change by selecting Maintenance > Reindex > Keyword file. Reindexing databases is important to ensure Statement Analyzer will properly run. Reindex databases individually or all of them every time changes are made.
Adding a Description
Let's now add a description for transitional words with a grammatical function of contrast that is meaningful in a context of detecting deception.

Make sure Statement Analyzer is not running and open ANALDESC.DBF using OpenOffice Calc. The first sheet contains every description Statement Analyzer has. There are two (2) columns available as previously explained. Move to the first available row, which should be around cell A102. Enter 102 in cell A102 and the following description in cell B102.
There are numerous transitional words with a grammatical function of contrast. When a subject uses words or phrases such as “however”, “in contrast”, “indeed”, “instead”, “nevertheless”, “on the contrary”, “on the other hand”, “yet” he or she is making a comparison among two or more thoughts. A deceptive person uses this technique to either minimize his or her involvement in a crime or maximize his or her potential benefits on the outcome of his or her deception.

After he or she finishes telling his or her story, come back to the contrasting thoughts that he or she has mentioned and make sure to understand which thoughts are more important. A subject may focus on or avoid one or another thought while retelling his or her story and it may tip off which is the most important.
Save once the description is added and close the database. Open Statement Analyzer, rebuild index files, enter the list of new keywords in a new statement, analyze, and make sure that the new keywords are all discovered and their description is properly displayed.

Text Processing Engine
The matching algorithm in Statement Analyzer is simple and reduced down to case insensitive matches. Matching phrases is the hardest and most complex task. For this reason, one may be willing to simplify keywords in order to maximize the hits returned by the software. “On one hand” and “on the other hand” could simply be the word “hand”. Or, every single entry can be entered provided that one has enough patience.

Tips
Be careful when editing databases and respect the file format, data types, character set and references to other records.

In OpenOffice, it is possible to activate and run a spell checker. It is useful on descriptions but remember that keywords are best as they currently are since misspelled words or manner of writing may be meaningful. Be careful when copying rich text into a cell in OpenOffice Calc because it may alter the format.

One may want to add every transitional word available. It is a good thing. However, it is not just about identifying the words, it is capital to understand when they are deceptive and when they are not.

No comments:

Post a Comment