Title: XKEYSCORE Workflows 2009

Release Date: 2015-07-01

Document Date: 2009-03-05

Description: This NSA presentation dated 5 March 2009 outlines the use of Workflows within XKeyScore and includes screenshots: see the Intercept article XKEYSCORE: NSA’s Google for the World’s Private Communications, 1 July 2015.

Document: xks-workflows-2009-p1-normal.gif:
XKEYSCORE

Workflows

05 March 2009

i3I

SECRET//COMINT//REL TO USA, AUS, CAN, GBR, MZL



i —*



DERIVED FROM: NSA/CSSM 1-52
DATED: 20070108
DECLASSIFY ON: 20320108

xks-workflows-2009-p2-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

What is a workflow?

Workflows automate queries.

> One-time
Standing

Every search type can be a workflow.
■ Same functionality and capability

Follow on actions

■ Email alert

□ Download actions

■ Metadata summary

SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p3-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

Who can submit a workflow? r ;T

Anyone!

One owner per workflow
Multiple-users can be notified
If ownership needs to be changed, a ticket
can be submitted to the team.

Future: sharing workflows
■ Right now, only the owner has the results in their
“My Results” view.

SECRET//COMINT//RELTO USA, AUS, CAN. GBR, NZLxks-workflows-2009-p4-normal.gif:
SECRET//COMINT//RELTO USA. AUS. CAN, GBR. NZL

What can I do with a workflow?

Workflows can be configured to run once
Workflows can be configured to run daily
- Every 1,2,3, 4, 6, 8, 12 or 24 hours
You can set an offset to start running at a certain
hour

Download results
Email results and email alerts
MAILORDER results
MySQL report

SECRET//COMINT//RELTO USA, AUS, CAN. GBR, NZLxks-workflows-2009-p5-normal.gif:
SECRET//COMINT//RELTO USA. AUS. CAN, GBR. NZL

Why do I want a workflow?

XKEYSCORE has a rolling buffer of data
Repetitive queries
Sigdev purpose

Fingerprint and appid testing
Queries take a long time during high times
Follow on actions
■ Google Earth data
□ Statistics

□ Customizable - write a script!

SECRET//COMINT//RELTO USA, AUS, CAN. GBR, NZLxks-workflows-2009-p6-normal.gif:
SECRET//C0MINT/7RELT0 USA, AUS, CAN, GBR, NZL



".I'M".

2

>

I

3

Q

2.

a>

§ a

a> &

O)



Cl

a

Û)xks-workflows-2009-p7-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

How do I setup a workflow?

w

First, s
workflc

SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p8-normal.gif:
SECRET//COMINT7/RELTO USA, AUS. CAN, GBR. N2L

How do I setup a workflow?

Workflow Central Request Wizard

Basic Information

Quety Name:

Query Xstrfcatco:
Addtonal Justification:
Mr anda Nurrfcsr:

Fnd_my_appd
Testng «pd agrature

ring or one-

ist be unique per user
must have a justification
justifications

Datetime: 11 Day 2009-03^)4 [3 00:00 C Step: 2009-03-05 [3 23:59 0 «

A ec cuing Seaich One lime Search)

Baue features Help

Runs once over
a set datetime
range

Cancel 4 Prev

► Next

SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p9-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. N2L

How do I setup a workflow?

f

Selec

searc

Workflow Central Request Wizard

Add Search Fields

Search values ¿re ANDed by default

To OR Search Helds:

* use the Multple Field Search tab (below the nput fields).

* Setect all the fields you wish to search.

To OR Search Values:

* Type \DR' between each value (no quotes).

See Search Value Hefc below fix rrcre details or
fix a description of boolean logic go to here.

Scorch Field

From IP A*fress Oft To P Address

Search Value
1 2.3 4

Rcwve

X

I To Port
snqte F»U Search
Search Value Help

Multiple Field Search

Cared ■! Prev

P Next

ant to



or every field,
du must select
le PLUS key

SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p10-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

Group by option

7

Group b
1 Red
Reti

Workflow Central Request Wizard
Group Search fields
Would you like to group any fields?

NO

i Yes

Group By Type

Table Urroue Values:

Global Unique Values:
Columns to Group By
Dateune:

Client IP (X-Fowarded-Foc):
Username:

Attrtoute lrrfo:

From IP Adi ess:

To IP Address:

From Port:

To Port:

From Country (IP):

To Country (IP):

From City (P):

To City (P):

From Latitude (IP):

C

C

r

r

r

c

r

r

n

r

r

r

r

r

r

.¿i_____________

Caned A Prev

d ita results.



This option groups paoh

and

OTartastetact&aJah’esettii&s.

concatenated.

Select the fields you
want to group by.

if1

> Next

SECRET//COMINTi7REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p11-normal.gif:
SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. NZL

Select databases

Workflow Central Request Wirord

Select the Database(s) to query

T xks- XlO C**CS. :CP)
r xks- iqsummary (xks-

r Content must exist

rqsummary)

1 &; owkfei |

[ UncheckÄ 1

Basic Features Help

If this is selected, results
are only returned if the
content still exists at site.

Cancel I Prev

> [Next

SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p12-normal.gif:
SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. N2L

Follow on Actions

All<

All<

loca

All<

Workflow Central Request Wizard
Follow on Action«

Would you like to add any follow on actions

No

Yes

Sow

Emai Alert
I Emal Alert
SQL Repat
Cowrtoad Sessoro

Sow Arownsrls

r Return Cnty With Resdts

intent) to another

Add



fiBtepèa^^ârtlreëtflÈcrtent

^attf^bba^feffljüsmpleted.

Carted * Prev

► Next

SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p13-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

Email alert

Workflow Central Request Wizard
I ollow-on Actions

Would you like to odd any follow on actions

''No

Y«

Script

Ema ¿teit

Script AjtyjrKrtz

AM

EmalTo:

ROWR:

r Return Orty With Resets

TTJ

Cared i Prev

* Next

Comma delimited email
addresses.

This option only sends an
email if you workflow has
results.

SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p14-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

SQL report

Workflow Central Request Wizard
Follow on Artiom

Would you like to add any follow on actions

Sotot

SQL Repcct

Sot* A/^mcfts
Type:
EmalTo:
Email Subject:
Email Content:
Email Attachment: ROWR: “ Emai Attachment r Return Orb/ With Results
Ftename:
Mai Order TrQraph:

SQL: SaECT FROM %{OUTPOT TAELE) WHERE GROUP 8Y ZA
G21P: r Compress Contents

CSV or HTML

Thi&friust be a VALID 3Qt^

si^Sfi?.tadata that a user

can set.

Example.

SELECT casenotation, sigad
FROM %{OUTPUT TABLE}
WHERE sigad!=‘’

GROUP BY casenotation

Cancel i Ptev

► Next

SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p15-normal.gif:
>10 01 101
1001 I 001 >00

SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. NZL

Download Results

r

Workflow Central Revest Wizard

Allions

NO

Yes

Would you like to odd any follow on action«

Script

Script Arguments

Dowrtoad Sessions

-

User ID:

EmalTa

Ernal subject:

Emal Content:

ROWR:

Flename:

Mai Oder
Trigraoh:

GZIP:

Send To Agiitv:

Cancel I Prev

AM



r Retun Only With Resdts

r Compress Contents
T Send To Açftty

► Hext



SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p16-normal.gif:
SECRET//COMINT//RELTO USA. AUS. CAN, GBR. N2L

You’re almost done!

w.

Workflow Central Request Wizard

Workflow Review

Tlits query (Pndjny_apod) *4 search the Full Log title n database^):
xks-jychanrqO

The query wf rvr CONTINUOUSLY executing every 6 hours begrmrtg at 5:00 EST
The query wil execute the fdowing search criteria:



From IP Addres$

1.2-3.4





To Porte/field>

< value >80 value >





ApplO (+F«igerprmts)*

search/google*



WorlflowXML

Cancel A Rrev

Soto*

SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p17-normal.gif:
>10 01 101
1001 I 001 >001

SECRET//COMINT7/RELTO USA. AUS. CAN, GBR. NZL

Workflow Pending

XKEYSCORE wefcom* I

Ö ».»• TS »»**>•£•>«■
M, Wllkbn

Mien •
ft-»,'*»

a M.C, trajnj^i

State a
pending

Actions

^XU/ Tf

a i

PV'd ■»





5 c* I



SECRET7/C0MINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p18-normal.gif:
SECRET//COMINT7/RELTO USA, AUS. CAN. GBR. N2L

Workflow Approved

SECRET//COMINT/7RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p19-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. N2L

Common mistakes

Workflow Central Request Wizard

From IP and To IP
with the same value.

In this view, terms are
ANDed together.

Use Multiple Field
Search Tab.

Add Search fields

Search Vabcs arc ANDed by dote*.

TO OR Search Reids:

* Use the Mjlopte Field Search tab (below the input fields).

* Select all the fields you wish to search.

To OR Search Values:

* Type CR' between each value (no quotes).

See Search value i-feb below for more details or
for a descrpncn of boolean togic go to here.

C«X«I i Pt*v

► Next

SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p20-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

Common mistakes

Using the multiple
field search does not
break this up into 3
searchvalue pairs.

Enter each term
separately in the
singe fieldsearch.

Workflow Central Request Wizard
Add Search Fields

Search values are ANDed by default

To OR Search Fields:

* Use the Multiple Field Search tab (be tow the «put fields).

* Select all the fields you wish to search.

To OR Search Values:

* Type W between each value (no quotes).

See Search Value be tow for more details or

for a description of boolean log»: go to here.

Search Value
1.2 34
5.678
80

Search Field
Frcm P Adcicss
To IP Address
Frcm Perl

Remove

Single Field Search Mutcir Fir d Se»ch
Search Value Help

Cancel "I Prev

> Next

SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL

-•XX*xks-workflows-2009-p21-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. NZL

Common mistakes

y

This will return ALL
casenotations.

a will be deafeted
by u!a” but a does
equal “!b”

All the defeated
values must be
ANDed together.

Workflow Central Request Wizard
Add Search Fields

Search Values are ANOed by default. To OR Search Fields: * Use the Muhipte Field Search tab (below the nput fields). * Select all the feds you wish to search. To OR Search Values: * Type 'OR' between each vab» (no quotes).
See Search value Help below for more details cr for a deserpton of boolean logi: go to here.
Seven Fiefcl Seven Value Remove
Casenototton 1« X
C«senot«tFy> to X
Casenototwn fc X
Coienct otton XI 1C 1
rU±J
Single Field Search yuo'e FKid Scorch



Con: el A Prev

► Next

SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p22-normal.gif:
SECRET//COMINT//RELTO USA, AUS. CAN, GBR. N2L

Common mistakes

/
Workflow Central Request Wizard is -

Add Search Fields

Search Values are ANDed by default To OR Search Fields: * Use ihe Multiple Feld Search tab (below the rput fetes). * Select all the fields you wish to search. To OR Search Values: * Type 'OR' between each valje (no quotes). See Search Valua Fte£ below for rae details or for a dascnptfcn of boolean logo go to here.
Search Flew Search vafue Re»*ove
CosenoUftcn fc X
Casercta&cn W X
1 SlOAD AJJC-993 * ■


Select the Databasc(s) to query

§-MJS silc*

•FS»
•NZsNtc

r Content must exist

•/ ¡Check All 1
UrxheckAI

Caret

If you are selecting
specific SIGADs, only
select the sites that
have data from that
SIGAD.

1 Queries will return
faster.

Slrlg tet£&GABcted
Less work for the

system.

u =
IT'

SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZLxks-workflows-2009-p23-normal.gif:
SECRET//CON1INT//RELTO USA, AUS. CAN, GBR. NZL

Common mistakes

If you select the
SQL Report option,
make sure you put a
valid SQL statement!

SQL statement filled in:

SELECT casenotation,
courfi(tf|pTY

FRcM^o^VpUTJTABLE}
WHERE casenotation!=‘’
GROUP BY casenotation

Workflow Central Request Wtfard

Follow-on Actions



‘ No
Yes
Sort Serf* Arjjncrti Add
Type: CSV
a?. Report V |
Emil To: inaVstCwork.com
Emil Stfcject: Mr Woridbw Resiits
Emal Center*: Bod SCL • empty
Emal Attadrnent: r Emol Attachment
ROVlR: r Return Only with Resiits
Flename:
Mai Order Trgroph:
“ ( SELECT casecotatcrv couitf*) *] \ FROM %{OnWT_TABlE} *J ) \v>ER£c«nenotatont“" . J vGROCP BY easenotatcnJ Zl.
GZiP:

Carved i Prev

SECRET//COMINT//RELTO USA, AUS, CAN, GBR, NZL

► Nextxks-workflows-2009-p24-normal.gif:


SECRET.7C0MINT//RELT0 USA. AUS. CAN. GBR. N2L