2. Purposes of Evaluation

• Evaluate a Civil Case

• Evaluate a Criminal Case

• Decide on a Criminal Action

• Compare Assertions

• Compare Potential Suspects

Four types of features apply to all purposes of evaluation:

• Judge Facts by Probabilities. Section 2.1 describes how to judge facts by probabilities.
  

• Evaluate Assertions. Section 2.2 describes how to evaluate assertions: There are two types of assertions (i.e., simple and compound), and each type of assertion can be evaluated in two ways (i.e., intuitively and logically).
  

• Compare Assertions. Section 2.3 describes how to decide by comparing assertions to each other or to a standard of proof.
  

• Evaluate A Case. Section 2.4 describes how to evaluate a case: Evaluation of a case involves evaluation of one or more assertions and one or more decisions resulting from the evaluations. A case can be evaluated preliminarily by the administrator (only) and precisely by multiple evaluators.
  

2.1 Judge Facts by Probabilities

Facts are judged to evaluate an assertion. If more than one fact is judged, the facts must be independent of each other. (Two facts are independent if knowing one fact does not change your judgment of the other fact.)

Depending upon the type of case, between two and four types of probabilities are relevant to each fact. All four types of probabilities are listed below. They are independent of each other. (Two types of probabilities are independent if knowing the probability of one type does not cause you to change the probability of the other type.) That is, the value of one type of probability implies nothing about the value of another. The first two types of probabilities depend only upon the fact, and the second two types of probabilities depend upon both the fact and the assertion being evaluated. For convenience, FactLogic sets the default value for the first type at 100% and the other type at 0%:

• Probability the Fact Will Be Admitted. This probability exists only for cases that may go to trial. The administrator enters this probability, and other evaluators are not aware that it exists. Because this probability depends only upon the fact, it is constant for all assertions and all evaluators.
  

• Probability the Fact Is True. This probability depends only upon the fact; it is independent of any assertion. Therefore, for each evaluator's judgments, this probability is constant for all assertions. A fact can be either intentionally inaccurate or unintentionally inaccurate:[2]
  
  ο Intentional Inaccuracy. A fact can be inaccurate due to falsehood, fraud, disguise, planted evidence, or other deceptions.
  ο Unintentional Inaccuracy. A fact can be inaccurate due to unreliable equipment, incorrect data, mislabeled data, degraded material, faulty senses, faulty memory, or other mistakes.

• Probability the Fact Proves the Assertion, Given the Fact Is True. This probability measures the ability of the fact to prove an assertion (assuming the fact is true). This ability depends upon both the fact and the assertion. A fact can be very probative for one assertion but not at all for another.[3]
  

• Probability the Fact Disproves the Assertion, Given the Fact Is True. This probability measures the ability of the fact to disprove an assertion (assuming the fact is true). This ability depends upon both the fact and the assertion. A fact can be very probative for one assertion but not at all for another. This probability measures the exculpatory nature of the fact. Of all probabilities, this one occurs least frequently, but it can have a significant effect on an assertion when it occurs.

For each fact, the evaluator should enter the probability the fact is true and any of the other types of probabilities that seem appropriate to the assertion. The probabilities may reveal strengths and weakness of the facts and, consequently, strengths or weaknesses of the assertion. For all types of cases, FactLogic combines all probabilities for all facts according to probability theory.[4]

The result is the logical probability that the assertion is true. (As will be discussed below, for cases that may go to trial, judgments of facts are also be combined intuitively – as was necessarily done before FactLogic showed how to combine judgments logically and as is done by fact finders in a trial.)

(Rectangles indicate entities, and ovals indicate actions.)

2.2 Evaluate Assertions

Assertions can be simple or compound. A simple assertion is an assertion that consists of a simple statement (e.g., a single idea). A compound assertion is an assertion that consists of multiple simple assertions. Evaluations of simple assertions and compound assertions can be intuitive or logical, depending on the application.

2.2.1 Evaluate A Simple Assertion

A simple assertion is a simple statement. It can be evaluated in two ways:

• Intuitive Evaluation. If judgments of the facts are not quantified, the judgments must be combined intuitively and the evaluation of the assertion is intuitive.
  

• Logical Evaluation. If the facts are independent, and if judgments of the facts are quantified as probabilities, the judgments are combined according to probability theory, and the evaluation of the assertion is logical.[5]
  

2.2.2 Evaluate A Compound Assertion

A compound assertion consists of multiple simple assertions. The following are two examples of compound assertions:

• Cause of Action. If the compound assertion is a cause of action in a legal case, the simple assertions are called the elements. For example, the cause of action, negligence, has three elements: Defendant had a duty; defendant caused the injury; and plaintiff was damaged.
  

• Hypotheses. If the compound assertion is a hypothesis, each simple assertion might incorporate one of the six interrogatories: who, what, when, where, what, and how.
   

A compound assertion is not evaluated directly; it is evaluated indirectly from the evaluations of its simple assertions. It can be evaluated in two ways:

• Intuitive Evaluation. If evaluations of the simple assertions are not quantified, the evaluations must be combined intuitively, and the evaluation of the compound assertion is intuitive.
  

• Logical Evaluation. If the simple assertions are independent, and if evaluations are quantified as probabilities, the evaluations are combined according to probability theory, and the evaluation of the compound assertion is logical.[6]
  

2.3 Compare Assertions

A decision results from a comparison. The comparison can be either between two assertions or between an assertion and a standard of proof.

2.3.1 Compare Two Assertions

Comparing two assertions is an excellent application of FactLogic, particularly if the assertions are hypotheses. To decide which is the more plausible hypothesis, evaluate each hypothetical assertion from the same facts, and compare the probabilities that each assertion is true. This application is especially useful for investigation, research, and predicting events (such as attacks or crimes). A common type of assertion is the responsibility of potential suspects.

2.3.2 Compare An Assertion to A Standard of Proof

In some applications, the evaluator decides on an action by comparing the probability an assertion is true to a standard of proof. That is, if the following difference,

(Probability Assertion Is True) – (Standard of Proof),

is positive, there is sufficient reason to take some action (which is related to the purpose of evaluation). The table, How To Use FactLogic, lists five purposes of evaluation: The first three purposes require a decision from comparing an assertion to a standard of proof, and the last two purposes require a decision from comparing assertions. The three standards of proof are:

• Preponderance of Evidence. The standard of proof for a civil case is preponderance of evidence. It is fixed by statute as any amount greater than 50%.
  

• Beyond a Reasonable Certainty. The standard of proof for a criminal case is stated as "beyond a reasonable doubt" (even though it should be stated as "beyond a reasonable certainty"). The fact finder is free to judge this value provided it is greater than 0% and less than 100%; several analysts report that the average from respondents is approximately 85%.
  

• Probable Cause. The standard of proof to decide on a criminal action is probable cause. It is usually considered any amount greater than 50%.

Other standards of proof are defined (e.g., reasonable suspicion), but they are not used here.

2.4 Evaluate A Case

For the purpose of evaluation, a case consists of one or more facts, one or more assertions, and, sometimes, a standard of proof. Cases can be evaluated either by the administrator in a preliminary way or by a number of evaluators to obtain precision for the evaluations.

2.4.1 Preliminary Evaluation (by the Administrator)

The administrator can evaluate the case preliminarily to conduct “sensitivity analysis” - to analyze how different probabilities about one or more facts would affect the probability the assertion is true. Because the administrator is the only evaluator, the evaluation of the assertion is simply “one person’s opinion,” and there can be no precision.

2.4.2 Precise Evaluation (by Multiple Evaluators)

You can invite evaluators to evaluate the case, thereby obtaining some precision for the estimate of the assertion. FactLogic computes statistics from these evaluations. A population is a group of evaluators that meet certain criteria (e.g., the venire in the Fulton County Judicial District). The expected value of all evaluations from the population is called the (true) mean. If it is either impractical or impossible to obtain evaluations from all evaluators in the population, a random sample of evaluators is sometimes obtained. A random sample is a random subset of the population (e.g., 31 of the 2,351 members of the venire in the Fulton County Judicial District). The expected value of all evaluations from the sample is called the average. Generally, the larger the random sample, the greater the precision.

FactLogic determines the achieved precision from the evaluations, and it also estimates the number of evaluations required to achieve a specified precision.

A. Achieved Precision

For each simple assertion, Factlogic computes the average of the sample and its precision (i.e., a measure of the closeness of the average of the sample to the mean of the population). For compound assertions, FactLogic computes the average and its precision from each of the simple assertions (that constitute it). Precision can be measured as the width of the 95% confidence interval that is centered on the average: The probability is 95% that both the average (of the sample) and the mean (of the population) are within the interval. Generally, the more evaluations, the greater is the precision. The achieved precision should be equal to the importance of the assertion. If it is less, the following information can help you achieve the precision you specify.

B. Number of Evaluators Required To Achieve A Specified Precision

If you obtain at least six evaluations for a simple assertion, FactLogic estimates the total number of evaluations that will achieve each of six selected precisions (hopefully, one is near the precision you might desire): Specifically, Factlogic determines the (total) number of evaluations that are required to be 95% confident that the difference between the average (of the sample) and the mean (of the population) is within a specified percentage. Suppose, for example, that the average probability the assertion true is 75%. FactLogic determines the number of evaluations required to achieve the following differences:

• Difference of 1%. FactLogic determines how many evaluations are required to be 95% confident that the true mean is within 1% of the average of 75% (i.e., between 74% and 76%);
  

• Diffference of 2%. FactLogic determines how many evaluations are required to be 95% confident that the true mean is within 2% of the average of 75% (i.e., between 73% and 77%);
  

• Etc. (for distances of 5%, 10%, 20%, and 50%).
  

Generally, the more precision you specify, the more evaluations will be required. The desired precision should be proportional to the importance of the assertion.

[2] A fact that is judged “less than true” needn’t be false (i.e., having a probability of 0%). It can be somewhat inaccurate (e.g., having a probability of, say, 90%).

[3] If a fact-assertion combination occurs often, this probability might not need to be judged because it is known from data. An example of knowing the probability from data would be the frequency of occurrence (i.e., the probability) of a certain type of injury to the body (i.e., the fact) to a possible victim of child abuse (i.e., the assertion).

[4] In mathematics, the word “theory” is not taken lightly. Probability theory is a set of theorems that are proved either from axioms or from other proved theorems. Theorems are conjured and then proved, not simply conjured.

[5] Two facts are independent if knowing one fact does not change your judgment of the other fact

[6] Two assertions are independent if knowing the evaluation of one assertion does not change your judgments of the facts for the other assertion.