|
|
@ -1,15 +1,15 @@ |
|
|
Iniitial testing scenario is based on the desire to anonymise contracts; these are real contracts that have been agreed between real people and/or organisations. While some of the sample set may in practice be neither confidential or sensitive, for the purposes of the exercise we will assume they are, and therefore a successful run will successfully remove confidential and/or identifying information from the contracts but leave them otherwise intact. In the case of the specific test case built the information removed will be replaced by "tags". |
|
|
Iniitial testing scenario is based on the desire to anonymise contracts; these are real contracts that have been agreed between real people and/or organisations. While some of the sample set may in practice be neither confidential or sensitive, for the purposes of the exercise we will assume they are, and therefore a successful run will successfully remove confidential and/or identifying information from the contracts but leave them otherwise intact. In the case of the specific test case built the information removed will be replaced by "tags". |
|
|
|
|
|
|
|
|
Test procedure: |
|
|
Test procedure: |
|
|
identify key items requiring anonymisation (names, dates, amounts?); |
|
|
|
|
|
list the manually processed and unprocessed docs by pair (noting the manually processed docs have as part of the anonymisation acquired neutral filenames); |
|
|
|
|
|
diff a document pair to see what content has been changed and use that to work out what search patterns are needed; |
|
|
|
|
|
select unprocessed half of pair and try initial search patterns and check whether the items are found, adjust as necessary; |
|
|
|
|
|
create replacement matrix for the items in the search pattern; |
|
|
|
|
|
process the doc; |
|
|
|
|
|
diff the resulting document with the manually processed one and assess any differences found; |
|
|
|
|
|
for any differences found that cannot be fixed by amendments to search pattern create issues in https://git.law/newroco/searchanddisplace-core/issues (noting that it is possible differences are because S&D is doing a better job than the manual process!); |
|
|
|
|
|
repeat for 9 more documents. |
|
|
|
|
|
|
|
|
* identify key items requiring anonymisation (names, dates, amounts?); |
|
|
|
|
|
* list the manually processed and unprocessed docs by pair (noting the manually processed docs have as part of the anonymisation acquired neutral filenames); |
|
|
|
|
|
* diff a document pair to see what content has been changed and use that to work out what search patterns are needed; |
|
|
|
|
|
* select unprocessed half of pair and try initial search patterns and check whether the items are found, adjust as necessary; |
|
|
|
|
|
* create replacement matrix for the items in the search pattern; |
|
|
|
|
|
* process the doc; |
|
|
|
|
|
* diff the resulting document with the manually processed one and assess any differences found; |
|
|
|
|
|
* for any differences found that cannot be fixed by amendments to search pattern create issues in https://git.law/newroco/searchanddisplace-core/issues (noting that it is possible differences are because S&D is doing a better job than the manual process!); |
|
|
|
|
|
* repeat for 9 more documents. |
|
|
|
|
|
|
|
|
If issues are found, repeat the process after code is updated, including at least 5 docs that have not previously been used. |
|
|
|
|
|
|
|
|
If issues are found, repeat the process after code is updated, including at least 5 docs that have not previously been used.<br> |
|
|
NB Once we have a tested good search pattern for the contract anonymisation process, we'll use it to generate a set of "changed identifier" docs which can then form a caucus for a completely open test set. |
|
|
NB Once we have a tested good search pattern for the contract anonymisation process, we'll use it to generate a set of "changed identifier" docs which can then form a caucus for a completely open test set. |
