From:	Slade Griffin
Sent:	Thursday, May 06, 2010 11:08 AM
To:	Bobby Brown
Cc:	Sandy Bacik
Subject:	Security Conformity Notes - 05/06/2010

Sandy and Bobby,

Here are the items I got down:
Updates: Kevin Brown is working on use cases for AMI

Discussion:

What type of requirements should be included? High-level requirements so that labs can develop their 
own methods.

Sandy's discussion/presentation

What types of things need to be in the reports? NDA, scope of work. Justin also suggested liability 
release which may be included in the statement of work or testing agreement. Additionally, lab isolation 
should be documented in the description of the testing environment.

Bruce stated that the product capabilities should be stated by the vendor/manufacturer.

Conformance testing should be pass/fail, possibly inconclusive, and not vulnerability research or a pen 
testing approach.
inconclusive means "has not failed" by default.

Report should include data classification (confidential, business sensitive, etc.) and be clearly marked as 
such

Data (report) belongs to the vendor and there should be three reports: Public, commercial, and private 
and the vendor will determine what to release in which format.

A baseline test should be performed which all testing labs must pass in order to certify a lab. Additionally, 
background information and "chain of certification" (who certified this testing lab) should be in the report.

Device configuration should be included in the report. 

Sufficient detail should exist to make findings in reports 100% reproducible in other labs.

Tester's notes will be supplied when there is a "fail" but not necessarily for objects that "pass."

Sandy will create an "overview of guidelines."

Bobby's working group:

existing PCI, ICSA, and other standards were observed in order to "glean the cream and scrap the crap"

It was suggested to not observe the common criteria as it analyzes process testing and that this testing 
should stick to "nuts and bolts" such as using the PCI "yes/no" yet include things like ICSA interoperability 
tables.

Some discussion ensued regarding whether a product should be run in a default configuration or 
configured according to vendor-supplied documentation. This segued into whether documentation should 
be "tested" as well.

The point-of-view from the group should be from the tester's perspective. 

Suggested tests included testing encryption via PCAP then running an entropy test. Running OWASP 
testing against IP-connected devices.

It was suggested that this group remain a high-level "advisory board" and not "get in the weeds." For 
example, do not define how a test is performed but what you expect to be tested.


I hope that's helpful,

Slade G