Title: XKEYSCORE Workflows 2009
Release Date: 2015-07-01
Document Date: 2009-03-05
Description: This NSA presentation dated 5 March 2009 outlines the use of Workflows within XKeyScore and includes screenshots: see the Intercept article XKEYSCORE: NSA’s Google for the World’s Private Communications, 1 July 2015.
Document: xks-workflows-2009-p1-normal.gif:
XKEYSCORE
Workflows
05 March 2009
i3I
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, MZL
i —*
DERIVED FROM: NSA/CSSM 1-52
DATED: 20070108
DECLASSIFY ON: 20320108
xks-workflows-2009-p2-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
What is a workflow?
Workflows automate queries.
> One-time
Standing
Every search type can be a workflow.
■ Same functionality and capability
Follow on actions
■ Email alert
□ Download actions
■ Metadata summary
SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p3-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
Who can submit a workflow? r ;T
Anyone!
One owner per workflow
Multiple-users can be notified
If ownership needs to be changed, a ticket
can be submitted to the team.
Future: sharing workflows
■ Right now, only the owner has the results in their
“My Results” view.
SECRET//COMINT//RELTO USA, AUS, CAN. GBR, NZLxks-workflows-2009-p4-normal.gif:
SECRET//COMINT//RELTO USA. AUS. CAN, GBR. NZL
What can I do with a workflow?
Workflows can be configured to run once
Workflows can be configured to run daily
- Every 1,2,3, 4, 6, 8, 12 or 24 hours
You can set an offset to start running at a certain
hour
Download results
Email results and email alerts
MAILORDER results
MySQL report
SECRET//COMINT//RELTO USA, AUS, CAN. GBR, NZLxks-workflows-2009-p5-normal.gif:
SECRET//COMINT//RELTO USA. AUS. CAN, GBR. NZL
Why do I want a workflow?
XKEYSCORE has a rolling buffer of data
Repetitive queries
Sigdev purpose
Fingerprint and appid testing
Queries take a long time during high times
Follow on actions
■ Google Earth data
□ Statistics
□ Customizable - write a script!
SECRET//COMINT//RELTO USA, AUS, CAN. GBR, NZLxks-workflows-2009-p6-normal.gif:
SECRET//C0MINT/7RELT0 USA, AUS, CAN, GBR, NZL
".I'M".
2
>
I
3
Q
2.
a>
§ a
a> &
O)
Cl
a
Û)xks-workflows-2009-p7-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
How do I setup a workflow?
w
First, s
workflc
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p8-normal.gif:
SECRET//COMINT7/RELTO USA, AUS. CAN, GBR. N2L
How do I setup a workflow?
Workflow Central Request Wizard
Basic Information
Quety Name:
Query Xstrfcatco:
Addtonal Justification:
Mr anda Nurrfcsr:
Fnd_my_appd
Testng «pd agrature
ring or one-
ist be unique per user
must have a justification
justifications
Datetime: 11 Day 2009-03^)4 [3 00:00 C Step: 2009-03-05 [3 23:59 0 «
A ec cuing Seaich One lime Search)
Baue features Help
Runs once over
a set datetime
range
Cancel 4 Prev
► Next
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p9-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. N2L
How do I setup a workflow?
f
Selec
searc
Workflow Central Request Wizard
Add Search Fields
Search values ¿re ANDed by default
To OR Search Helds:
* use the Multple Field Search tab (below the nput fields).
* Setect all the fields you wish to search.
To OR Search Values:
* Type \DR' between each value (no quotes).
See Search Value Hefc below fix rrcre details or
fix a description of boolean logic go to here.
Scorch Field
From IP A*fress Oft To P Address
Search Value
1 2.3 4
Rcwve
X
I To Port
snqte F»U Search
Search Value Help
Multiple Field Search
Cared ■! Prev
P Next
ant to
or every field,
du must select
le PLUS key
SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p10-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
Group by option
7
Group b
1 Red
Reti
Workflow Central Request Wizard
Group Search fields
Would you like to group any fields?
NO
i Yes
Group By Type
Table Urroue Values:
Global Unique Values:
Columns to Group By
Dateune:
Client IP (X-Fowarded-Foc):
Username:
Attrtoute lrrfo:
From IP Adi ess:
To IP Address:
From Port:
To Port:
From Country (IP):
To Country (IP):
From City (P):
To City (P):
From Latitude (IP):
C
C
r
r
r
c
r
r
n
r
r
r
r
r
r
.¿i_____________
Caned A Prev
d ita results.
This option groups paoh
and
OTartastetact&aJah’esettii&s.
concatenated.
Select the fields you
want to group by.
if1
> Next
SECRET//COMINTi7REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p11-normal.gif:
SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. NZL
Select databases
Workflow Central Request Wirord
Select the Database(s) to query
T xks- XlO C**CS. :CP)
r xks- iqsummary (xks-
r Content must exist
rqsummary)
1 &; owkfei |
[ UncheckÄ 1
Basic Features Help
If this is selected, results
are only returned if the
content still exists at site.
Cancel I Prev
> [Next
SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p12-normal.gif:
SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. N2L
Follow on Actions
All<
All<
loca
All<
Workflow Central Request Wizard
Follow on Action«
Would you like to add any follow on actions
No
Yes
Sow
Emai Alert
I Emal Alert
SQL Repat
Cowrtoad Sessoro
Sow Arownsrls
r Return Cnty With Resdts
intent) to another
Add
fiBtepèa^^ârtlreëtflÈcrtent
^attf^bba^feffljüsmpleted.
Carted * Prev
► Next
SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p13-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
Email alert
Workflow Central Request Wizard
I ollow-on Actions
Would you like to odd any follow on actions
''No
Y«
Script
Ema ¿teit
Script AjtyjrKrtz
AM
EmalTo:
ROWR:
r Return Orty With Resets
TTJ
Cared i Prev
* Next
Comma delimited email
addresses.
This option only sends an
email if you workflow has
results.
SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p14-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
SQL report
Workflow Central Request Wizard
Follow on Artiom
Would you like to add any follow on actions
Sotot
SQL Repcct
Sot* A/^mcfts
Type:
EmalTo:
Email Subject:
Email Content:
Email Attachment: ROWR: “ Emai Attachment r Return Orb/ With Results
Ftename:
Mai Order TrQraph:
SQL: SaECT FROM %{OUTPOT TAELE) WHERE GROUP 8Y ZA
G21P: r Compress Contents
CSV or HTML
Thi&friust be a VALID 3Qt^
si^Sfi?.tadata that a user
can set.
Example.
SELECT casenotation, sigad
FROM %{OUTPUT TABLE}
WHERE sigad!=‘’
GROUP BY casenotation
Cancel i Ptev
► Next
SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p15-normal.gif:
>10 01 101
1001 I 001 >00
SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. NZL
Download Results
r
Workflow Central Revest Wizard
Allions
NO
Yes
Would you like to odd any follow on action«
Script
Script Arguments
Dowrtoad Sessions
-
User ID:
EmalTa
Ernal subject:
Emal Content:
ROWR:
Flename:
Mai Oder
Trigraoh:
GZIP:
Send To Agiitv:
Cancel I Prev
AM
r Retun Only With Resdts
r Compress Contents
T Send To Açftty
► Hext
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p16-normal.gif:
SECRET//COMINT//RELTO USA. AUS. CAN, GBR. N2L
You’re almost done!
w.
Workflow Central Request Wizard
Workflow Review
Tlits query (Pndjny_apod) *4 search the Full Log title n database^):
xks-jychanrqO
The query wf rvr CONTINUOUSLY executing every 6 hours begrmrtg at 5:00 EST
The query wil execute the fdowing search criteria:
From IP Addres$
1.2-3.4
To Porte/field>
< value >80 value >
ApplO (+F«igerprmts)*
search/google*
WorlflowXML
Cancel A Rrev
Soto*
SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p17-normal.gif:
>10 01 101
1001 I 001 >001
SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. NZL
Workflow Pending
XKEYSCORE wefcom* I
Ö ».»• TS »»**>•£•>«■
M, Wllkbn
Mien •
ft-»,'*»
a M.C, trajnj^i
State a
pending
Actions
^XU/ Tf
a i
PV'd ■»
5 c* I
SECRET7/C0MINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p18-normal.gif:
SECRET//COMINT7/RELTO USA, AUS. CAN. GBR. N2L
Workflow Approved
SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p19-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. N2L
Common mistakes
Workflow Central Request Wizard
From IP and To IP
with the same value.
In this view, terms are
ANDed together.
Use Multiple Field
Search Tab.
Add Search fields
Search Vabcs arc ANDed by dote*.
TO OR Search Reids:
* Use the Mjlopte Field Search tab (below the input fields).
* Select all the fields you wish to search.
To OR Search Values:
* Type CR' between each value (no quotes).
See Search value i-feb below for more details or
for a descrpncn of boolean togic go to here.
C«X«I i Pt*v
► Next
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p20-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
Common mistakes
Using the multiple
field search does not
break this up into 3
searchvalue pairs.
Enter each term
separately in the
singe fieldsearch.
Workflow Central Request Wizard
Add Search Fields
Search values are ANDed by default
To OR Search Fields:
* Use the Multiple Field Search tab (be tow the «put fields).
* Select all the fields you wish to search.
To OR Search Values:
* Type W between each value (no quotes).
See Search Value be tow for more details or
for a description of boolean log»: go to here.
Search Value
1.2 34
5.678
80
Search Field
Frcm P Adcicss
To IP Address
Frcm Perl
Remove
Single Field Search Mutcir Fir d Se»ch
Search Value Help
Cancel "I Prev
> Next
SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
-•XX*xks-workflows-2009-p21-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL
Common mistakes
y
This will return ALL
casenotations.
a will be deafeted
by u!a” but a does
equal “!b”
All the defeated
values must be
ANDed together.
Workflow Central Request Wizard
Add Search Fields
Search Values are ANOed by default. To OR Search Fields: * Use the Muhipte Field Search tab (below the nput fields). * Select all the feds you wish to search. To OR Search Values: * Type 'OR' between each vab» (no quotes).
See Search value Help below for more details cr for a deserpton of boolean logi: go to here.
Seven Fiefcl Seven Value Remove
Casenototton 1« X
C«senot«tFy> to X
Casenototwn fc X
Coienct otton XI 1C 1
rU±J
Single Field Search yuo'e FKid Scorch
Con: el A Prev
► Next
SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p22-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. N2L
Common mistakes
/
Workflow Central Request Wizard is -
Add Search Fields
Search Values are ANDed by default To OR Search Fields: * Use ihe Multiple Feld Search tab (below the rput fetes). * Select all the fields you wish to search. To OR Search Values: * Type 'OR' between each valje (no quotes). See Search Valua Fte£ below for rae details or for a dascnptfcn of boolean logo go to here.
Search Flew Search vafue Re»*ove
CosenoUftcn fc X
Casercta&cn W X
1 SlOAD AJJC-993 * ■
Select the Databasc(s) to query
§-MJS silc*
•FS»
•NZsNtc
r Content must exist
•/ ¡Check All 1
UrxheckAI
Caret
If you are selecting
specific SIGADs, only
select the sites that
have data from that
SIGAD.
1 Queries will return
faster.
Slrlg tet£&GABcted
Less work for the
system.
u =
IT'
SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p23-normal.gif:
SECRET//CON1INT//RELTO USA, AUS. CAN, GBR. NZL
Common mistakes
If you select the
SQL Report option,
make sure you put a
valid SQL statement!
SQL statement filled in:
SELECT casenotation,
courfi(tf|pTY
FRcM^o^VpUTJTABLE}
WHERE casenotation!=‘’
GROUP BY casenotation
Workflow Central Request Wtfard
Follow-on Actions
‘ No
Yes
Sort Serf* Arjjncrti Add
Type: CSV
a?. Report V |
Emil To: inaVstCwork.com
Emil Stfcject: Mr Woridbw Resiits
Emal Center*: Bod SCL • empty
Emal Attadrnent: r Emol Attachment
ROVlR: r Return Only with Resiits
Flename:
Mai Order Trgroph:
“ ( SELECT casecotatcrv couitf*) *] \ FROM %{OnWT_TABlE} *J ) \v>ER£c«nenotatont“" . J vGROCP BY easenotatcnJ Zl.
GZiP:
Carved i Prev
SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZL
► Nextxks-workflows-2009-p24-normal.gif:
SECRET.7C0MINT//RELT0 USA. AUS. CAN. GBR. N2L

Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p1-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p2-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p3-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p4-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p5-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p6-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p7-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p8-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p9-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p10-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p11-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p12-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p13-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p14-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p15-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p16-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p17-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p18-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p19-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p20-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p21-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p22-normal.gif)
Download Document
XKEYSCORE Workflows 2009 (xks-workflows-2009-p23-normal.gif)