Update 'Testing process'

master
putt1ck 5 years ago
parent
commit
b9d3e6788c
  1. 20
      Testing-process.md

20
Testing-process.md

@ -1,15 +1,15 @@
Iniitial testing scenario is based on the desire to anonymise contracts; these are real contracts that have been agreed between real people and/or organisations. While some of the sample set may in practice be neither confidential or sensitive, for the purposes of the exercise we will assume they are, and therefore a successful run will successfully remove confidential and/or identifying information from the contracts but leave them otherwise intact. In the case of the specific test case built the information removed will be replaced by "tags".
Test procedure:
identify key items requiring anonymisation (names, dates, amounts?);
list the manually processed and unprocessed docs by pair (noting the manually processed docs have as part of the anonymisation acquired neutral filenames);
diff a document pair to see what content has been changed and use that to work out what search patterns are needed;
select unprocessed half of pair and try initial search patterns and check whether the items are found, adjust as necessary;
create replacement matrix for the items in the search pattern;
process the doc;
diff the resulting document with the manually processed one and assess any differences found;
for any differences found that cannot be fixed by amendments to search pattern create issues in https://git.law/newroco/searchanddisplace-core/issues (noting that it is possible differences are because S&D is doing a better job than the manual process!);
repeat for 9 more documents.
* identify key items requiring anonymisation (names, dates, amounts?);
* list the manually processed and unprocessed docs by pair (noting the manually processed docs have as part of the anonymisation acquired neutral filenames);
* diff a document pair to see what content has been changed and use that to work out what search patterns are needed;
* select unprocessed half of pair and try initial search patterns and check whether the items are found, adjust as necessary;
* create replacement matrix for the items in the search pattern;
* process the doc;
* diff the resulting document with the manually processed one and assess any differences found;
* for any differences found that cannot be fixed by amendments to search pattern create issues in https://git.law/newroco/searchanddisplace-core/issues (noting that it is possible differences are because S&D is doing a better job than the manual process!);
* repeat for 9 more documents.
If issues are found, repeat the process after code is updated, including at least 5 docs that have not previously been used.
If issues are found, repeat the process after code is updated, including at least 5 docs that have not previously been used.<br>
NB Once we have a tested good search pattern for the contract anonymisation process, we'll use it to generate a set of "changed identifier" docs which can then form a caucus for a completely open test set.
Loading…
Cancel
Save