Documentation

The Lumen API allows 3rd parties to search the database and, if the 3rd party has an submitter authentication token, submit new notices to Lumen Database.

Searches are disabled unless you have an authentication token. Please see Authentication for information about searching with an authentication token.

The sample code below is provided for your convenience but will need to be modified for your use case. Unfortunately Lumen is unable to offer programming assistance.

Authentication

Requests for notice data or searches through the API are not permitted without an Authentication Token. Submissions of data or notice creation through the API must be authenticated by including an Authentication Token.

With an authentication token, data requests are still throttled, but at a much higher limit (approximately one request per second). This limit was chosen to protect server resources while rarely or never interfering with most researchers' uses. However, if you are making a large number of automated requests, please sleep for at least one second between requests to avoid hitting the cap.

The Authentication Token may be included as a URL Parameter, an HTTP Request Header, or (when submitting data) as form data or JSON data.

Examples: Getting Data from Lumen

• URL Parameter: authentication_token

curl -kv -H "User-Agent: SomeUserAgentHere" https://lumendatabase.org/notices/1.json?authentication_token=abc123

curl -kv -H "User-Agent: SomeUserAgentHere" https://lumendatabase.org/notices/search.json?authentication_token=abc123

curl -kv -H "User-Agent: SomeUserAgentHere" "https://lumendatabase.org/notices/search.json?sender_name=James+Tiberius+Kirk&authentication_token=abc123"

• HTTP Request Header: X-Authentication-Token

curl -kv -H "User-Agent: SomeUserAgentHere" -H "X-Authentication-Token: abc123" https://lumendatabase.org/notices/1.json

curl -kv -H "User-Agent: SomeUserAgentHere" -H "X-Authentication-Token: abc123" https://lumendatabase.org/notices/search.json

Examples: Sending Data to Lumen

• URL Parameter: authentication_token

curl -H "User-Agent: SomeUserAgentHere" \
     -H "Accept: application/json" \
     -H "Content-type: application/json" \
     -d '{ "notice": {...} }' \
     'https://lumendatabase.org/notices?authentication_token=abc123'

• Request Header: X-Authentication-Token

curl -kv \
     -H "User-Agent: SomeUserAgentHere" \
     -H "X-Authentication-Token: abc123" \
     -F "notice[type]=Defamation" \
     https://lumendatabase.org/notices

curl -H "User-Agent: SomeUserAgentHere" \
     -H "Accept: application/json" \
     -H "Content-type: application/json" \
     -H "X-Authentication-Token: abc123" \
     -d '{ "notice": {...} }' \
     https://lumendatabase.org/notices

Troubleshooting

Did you get an HTTP 429 (Too Many Requests) response and a message saying you were browsing too fast?

Your API token is either missing or incorrectly applied. Check your request against the examples above.

Did you get a 'command not found' error using curl on the command line?

You may have to put the URL in quotes if it contains special characters.

Did you get an HTTP 401 (Unauthorized) response?

Your API token is either missing or incorrectly applied. Check your request against the examples above.

Getting an API token

Email [e-mail] and describe your use case. Lumen API tokens are intended for research use only; please read the API Terms of Use before requesting a token.

Request a Notice

Method: GET

Endpoint: https://lumendatabase.org/notices/<notice id>.json

Example Request

curl -H "User-Agent: SomeUserAgentHere" https://lumendatabase.org/notices/1.json

⚠️ Note: a custom user agent is required. Default user agents such as "curl" are blocked by our servers.

You may have to put the URL in quotes if it contains special characters.

Successful Responses

Return a JSON-encoded representation of selected notice attributes. Notice Types will have mapped attributes applied, and be under a root key articulating their type.

Example Successful Response

{
  "dmca":{
    "id":1,
    "title":"Lion King on YouTube",
    "body":null,
    "date_sent":"2013-06-04T19:23:12Z",
    "date_received":"2013-06-05T20:31:44Z",
    "topics":[
      "Anticircumvention (DMCA)",
      "Bookmarks",
      "Lumen"
    ],
    "tags": [
      "tag_1",
      "tag_2"
    ],
    "jurisdictions": [
      "US",
      "CA"
    ],
    "action_taken": "Partial",
    "sender_name": "Joe Lawyer",
    "recipient_name": "Google, Inc.",
    "works": [
      {
        "description": "Lion King Video",
        "copyrighted_urls": [
          { "url": "http://www.example.com/lion_king.mp4" },
          { "url": "http://www.example.com/lion_king.mov" }
        ],
        "infringing_urls": [
          { "url": "http://www.example.com/infringing1" },
          { "url": "http://www.example.com/infringing2" },
          { "url": "http://www.example.com/infringing3" }
        ]
      }
    ]
  }
}

In cases where infringing or copyrighted URLs were not submitted to Lumen, the associated field will say [{ "url": "No URL submitted" }].

Unsuccessful Responses

Return a 404 HTTP status header.

Request a list of Topics

Method: GET

Endpoint: https://lumendatabase.org/topics.json

curl -H "User-Agent: SomeUserAgentHere" https://lumendatabase.org/topics.json

Successful Responses

Return a JSON-encoded array of topics, including the following attributes:

Field	Type	Description
id	Integer	The unique ID used for the topic_ids array during notice creation.
name	String	The topic name
parent_id	Integer	The parent topic_id of this topic, or "null" if this is a root topic.

Example Successful Response

{
  "topics": [
    {
      "id": 1,
      "name": "Lumen",
      "parent_id": null
    },
    {
      "id": 2,
      "name": "Copyright",
      "parent_id": null
    },
    {
      "id": 3,
      "name": "Anticircumvention (DMCA)",
      "parent_id": 2
    }
  ]
}

Search notices via fulltext

The Lumen website allows for paginatable full-text searches of notices and relevant metadata. Results are sorted with the most relevant at the top. Notice search results contain the same data as an individually-requested Notice, with an additional "score" field. "score" articulates how "relevant" this result is to the query term. Higher numbers are more relevant.

With an API token, results can be extensive; consider requesting gzipped content (if using curl, add -H "Accept-Encoding: gzip").

Lumen is unable to return past the 10,000th result due to limitations in Elasticsearch. This means that querying deeper than page=1000 (with the default per_page of 10) will fail.

Terms are joined with an 'OR' by default.

Method: GET

Endpoint: https://Lumendatabase.org/notices/search?term=url_escaped_query&sender_name=Joe%20Smith&page=1

Parameters

Parameters	Description
term	The full-text query term. To perform exact searches, enclose your search term in double quotes (""). This ensures that the search engine looks for the exact phrase as it is written. Searching for "full-text query" will return results containing precisely this phrase.
term-require-all	If present, all words in the term query are required for a notice to be considered a match.
title	Search in the title field
title-require-all	If present, all words in the title query are required for a notice to be considered a match.
topics	Search within a notice's topics
topics-require-all	If present, all words in the topics query are required for a notice to be considered a match.
tags	Search within a notice's tags
tags-require-all	If present, all words in the tags query are required for a notice to be considered a match.
jurisdictions	Search within a notice's jurisdictions
jurisdictions-require-all	If present, all words in the jurisdictions query are required for a notice to be considered a match.
sender_name	Search in the sender's name
sender_name-require-all	If present, all words in the sender_name query are required for a notice to be considered a match.
principal_name	Search in the principal's name
principal_name-require-all	If present, all words in the principal_name query are required for a notice to be considered a match.
recipient_name	Search in the recipient's name
recipient_name-require-all	If present, all words in the recipient_name query are required for a notice to be considered a match.
works	Search within a work's description
works-require-all	If present, all words in the works query are required for a notice to be considered a match.
action_taken	Search based on the action taken on a notice.
entities_country_codes	Search within country codes of all notice's entities
topic_facet	Filter on topics facet
sender_name_facet	Filter on sender_name facet
principal_name_facet	Filter on principal_name facet
recipient_name_facet	Filter on recipient_name facet
tag_list_facet	Filter on a tag
country_code_facet	Filter on the submitter's country code
language_facet	Filter on the notice language code
action_taken_facet	Filter on the action_taken facet
date_received_facet	The date range (in milliseconds since the unix epoch) separated by "..", e.g. 1546344000000..1546603200000, see Epoch format
date_submitted	The date range (in milliseconds since the unix epoch) separated by "..", e.g. 1546344000000..1546603200000, see Epoch format
page	The page you're requesting - defaults to the first page of results.
per_page	The number of results per page. Defaults to 10.
sort_by	One of date_received asc, date_received desc, relevancy asc, or relevancy desc. Defaults to relevancy asc.

*-require-all Parameters

Let's say we're searching for notices with "George Jetson" in the title. By default, the search engine will search for notices with "George OR Jetson".

If you include 'title-require-all=yes' in your query, then the search engine will search for notices with "George AND Jetson" in the title, narrowing down your results considerably.

The various *-require-all parameters need a non-null value to be enabled - "true" or "yes" are both acceptable.

Facets

See below for more information about available facets. You can get an idea of how facets are formatted by submitting facet-less fulltext queries and then inspecting the Facets metadata returned by the search.

Troubleshooting very large searches

Searches with a very large number of results (e.g. "DMCA") may fail. If this happens, you will see the URL for your search in the address bar, but a "REQUEST REFUSED (500)" error in the page.

Fix this by narrowing your search. For instance, add a date facet to the end of the URL, such as &date_received_facet=1586404800000.0..1602216000000.0. This example yields results from April 9, 2020, through October 9, 2020. See Epoch format below for how to generate good numbers for this facet, or simply append this example to your URL and you should be taken back to a search results page which you can then tweak as you wish.

Successful Responses

Return a JSON-encoded hash including an array of notices and metadata about the search results.

Field	Type	Description
notices	Array	An array of Notices encoded as JSON data structures.
meta	Hash	Search Metadata about the results of the search. See Search Metadata.

Search Metadata

Field	Type	Description
query	Query	The search query meta information. See Query.
facets	Facet	How the result set "falls" into metadata facets. See Facets
current_page	Integer	The page number of the current set of results.
next_page	Integer	The page number of the next set of results, or null if this is the last page.
offset	Integer	How many total results in the result set before the current list of results.
per_page	Integer	Number of results per page
previous_page	Integer	The page number of the previous set of results, or null if this is the first page.
total_entries	Integer	The total number of results for the search query.
total_pages	Integer	The total number of pages in this result set.

Query

Field	Type	Description
term	String	The full-text search term
sender_name_facet	String	The sender_name value if this facet was submitted in this search.
recipient_name_facet	String	The recipient_name value if this facet was submitted in this search.
topic_facet	String	The topic value if this facet was submitted in this search.
date_received_facet	String	The date range used (in milliseconds since the unix epoch) separated by "..", e.g. 1546344000000..1546603200000, see Epoch format
tag_list_facet	String	The tag_list value if this facet was submitted in this search.
country_code_facet	String	The country_code value if this facet was submitted in this search.
language_facet	String	The language value if this facet was submitted in this search.

Facet

Facets aggregate documents along specific metadata. See the elasticsearch documentation for more information on facets.

Currently we're using value and range facets.

Facet	Facet Type	Description
sender_name_facet	Terms	The top 10 sender_names in this result set.
recipient_name_facet	Terms	The top 10 recipient_names in this result set.
topic_facet	Terms	The top 10 topics in this result set.
tag_list_facet	Terms	The top 10 tags in this result set.
country_code_facet	Terms	The top 10 country_codes in this result set.
language_facet	Terms	The top 10 languages in this result set.
date_received	Range	The available date range facets.

Terms Facet

Only the most relevant metadata is described below. See the elasticsearch documentation for more information. We only return the 10 most populous facets.

Field	Type	Description
total	Integer	The total number of results that can be faceted on this term and query.
other	Integer	The number of results not included in the facets returned.
terms	Array	An array including the term and count of items that are in that term facet.

Range Facet

Field	Type	Description
from	Integer	The start of this facet, in unix epoch milliseconds, e.g. 1546344000000, see Epoch format
to	Integer	The end of this facet, in unix epoch milliseconds, e.g. 1546603200000, see Epoch format
from_str	String	A textual representation of the start of this facet.
to_str	String	A textual representation of the end of this facet.
count	Integer	The number of documents that fall in this facet.

Example Successful Response

Note: Some facet information has been omitted to keep examples brief.

curl -H "User-Agent: SomeUserAgentHere" -H "Accept: application/json" -H "Content-type: application/json" 'https://lumendatabase.org/notices/search?term=star'

{
  "notices": [
    {
      "id": "491",
      "type": "DMCA",
      "title": "Star Wars",
      "body": null,
      "date_received": null,
      "language": "en",
      "topics": [
        "Trademark",
        "DMCA Notices",
        "John Doe Anonymity"
      ],
      "sender_name": "Joe Lawyer",
      "recipient_name": "Google, Inc.",
      "tags": [
        "tag_1",
        "tag_2"
      ],
      "jurisdictions": [
        "US",
        "CA"
      ],
      "works": [
        {
          "description": "Lion King Video",
          "copyrighted_urls": [
            { "url": "http://www.example.com/lion_king.mp4" },
            { "url": "http://www.example.com/lion_king.mov" }
          ],
          "infringing_urls": [
            { "url": "http://www.example.com/infringing1" },
            { "url": "http://www.example.com/infringing2" },
            { "url": "http://www.example.com/infringing3" }
          ]
        }
      ],
      "score": 0.64636606
    }
  ],
  "meta": {
    "query": {
      "term": "star"
    },
    "facets": {
      "sender_name_facet": {
        "_type": "terms",
        "missing": 0,
        "total": 7,
        "other": 0,
        "terms": [
          {
            "term": "Steve Simpson",
            "count": 5
          },
          {
            "term": "Mike Itten",
            "count": 2
          }
        ]
      },
      "recipient_name_facet": {
        "_type": "terms",
        "missing": 0,
        "total": 7,
        "other": 0,
        "terms": [
          {
            "term": "Google",
            "count": 5
          },
          {
            "term": "Twitter",
            "count": 2
          }
        ]
      },
    },
    "current_page": 1,
    "next_page": null,
    "offset": 0,
    "per_page": 10,
    "previous_page": null,
    "total_entries": 1,
    "total_pages": 1
  }
}

Example Unsuccessful Response

{
  "notices": [ ],
  "meta": {
    "query": {
      "term": "nonexistent"
    },
    "current_page": 1,
    "next_page": null,
    "offset": 0,
    "per_page": 10,
    "previous_page": null,
    "total_entries": 0,
    "total_pages": 0
  }
}

Search for Entities via fulltext

The Lumen database allows for paginatable full-text searches of Entities, useful to select existing entities during notice creation. This method is only available via the API, not the web site. For information about pagination metadata, please see Search notices via fulltext.

Method: GET

Endpoint: https://lumendatabase.org/entities/search.json?term=url_escaped_query&page=1

Parameters

Parameters	Description
term	The full-text query term
page	The page you're requesting - defaults to the first page of results.
per_page	The number of results per page. Defaults to 10.

Successful Responses

Return a JSON-encoded hash including an array of entities and metadata about the search results.

Field	Type	Description
entities	Array	An array of Entities encoded as JSON data structures.
meta	Hash	Search Metadata about the results of the search. See Search Metadata.

Entity

Field	Type	Description
id	string	The unique ID
parent_id	string	The parent ID or "null" if this is a "root" entity.
name	string	Full name
address_line_1	string
address_line_2	string
state	string
country_code	string	Ideally, an ISO country code.
phone	string
email	string
url	string
city	string

Example Successful Response

curl -H "User-Agent: SomeUserAgentHere" -H "Accept: application/json" -H "Content-type: application/json" 'https://lumendatabase.org/entities/search?term=joe&page=1'

{
  "entities": [
    {
      "id": "6",
      "parent_id": null,
      "name": "Steve Simpson",
      "address_line_1": "23 My St.",
      "address_line_2": null,
      "state": "CA",
      "country_code": "US",
      "phone": null,
      "email": null,
      "url": null,
      "city": "Scranton"
    }
  ],
  "meta": {
    "query": {
      "term": "simpson"
    },
    "facets": null,
    "current_page": 1,
    "next_page": null,
    "offset": 0,
    "per_page": 10,
    "previous_page": null,
    "total_entries": 1,
    "total_pages": 1
  }
}

Example Unsuccessful Response

{
  "entities": [ ],
  "meta": {
    "query": {
      "term": "nonexistent"
    },
    "facets": null,
    "current_page": 1,
    "next_page": null,
    "offset": 0,
    "per_page": 10,
    "previous_page": null,
    "total_entries": 0,
    "total_pages": 0
  }
}

Create notice

Submits a new Notice to the Lumen system.

There are several different "Notice Types"; these are essentially subclasses of the abstract Notice class. "Notice Types" allow us to track notice-specific attributes and create serialized representations with logical attribute names. Please see Notice Type Mapping, which defines what attributes are remapped for each Notice Type.

An authentication token is required to submit via the API. Please ask Lumen staff for your authentication token. Use of the the authentication token is described in the Authentication section.

Method: POST

Endpoint: https://lumendatabase.org/notices

Request

Field	Type	Description
notice	Notice	Required
authentication_token	string	See Authentication

Notice

Note: The notice types must be sent exactly as written here, i.e., CamelCase with no spaces except DMCA which is all-caps.

Field	Type	Description
title	string	Required
type	string	Required: one of 'Counternotice', 'CourtOrder', 'DataProtection', 'Defamation', 'DMCA', 'LawEnforcementRequest', 'Other', 'PrivateInformation', 'GovernmentRequest', or 'Trademark'
subject	string	Optional. A short description of the notice and its contents. E.g. "DMCA Notice Regarding Photographs" or "Court Order From Paris Court Re: Hate Speech"
body	string	Optional. Use this field to submit any additional text provided by the complainant that does not belong properly in any other notice field. No sensitive information or PII should be included here. E.g., Phone numbers, street addresses, allegedly defamatory language, etc.
date_sent	string	Any parseable time value, e.g. "2013-05-21", "2013-05-21 10:01:01 -04:00"
language	string	A two character language code. See "Language" below for the list.
date_received	string	Any parseable time value, e.g. "2013-05-21", "2013-05-21 10:01:01 -04:00"
source	string	How did you receive this notice - mail? online form?
topic_ids	[int]	An array of integers representing topic_ids - See Request a list of Topics.
tag_list	string	Comma separated tags, spaces are OK. Automatically lowercased.
regulation_list	string	Comma separated regulations / laws relevant to this notice, spaces are OK. Automatically lowercased. Only available for CourtOrder, and LawEnforcementRequest notice types.
jurisidiction_list	string	Comma separated list of jurisdictions, spaces are OK.
action_taken	string	One of 'Yes', 'No', or 'Partial'.
url_count	string	This field is used to indicate the total number of URLs contained in a particular Data Protection request. I.e. If the requester asked for the removal of 10 URLs, this should be set to "10". If no value is set, this will display as "unspecified".
request_type	string	One of 'Agency', 'Civil Subpoena', 'Email', 'Records Preservation', 'Subpoena', 'Warrant'. Valid only for LawEnforcementRequest notices.
mark_registration_number	string	A mark registration number. Valid only for Trademark notices.
works_attributes	[Work]	A list of Works
entity_notice_roles_attributes	[EntityNoticeRole]	A list of EntityNoticeRoles
file_uploads_attributes	[FileUpload]	A list of FileUploads
case_id_number	int	Optional, a court case number, specific to the CourtOrder type

Work

Field	Type	Description
copyrighted_urls_attributes	[CopyrightedUrl]	List of URLs that represent the original work.
kind	string	Required. Book, movie, video, etc.
description	string	Description of the work, which may include the copyright holder information
infringing_urls_attributes	[InfringingUrl]	List of URLs which infringe on the work

CopyrightedURL

Field	Type	Description
url	string	A URL that represents the original work. 8 kilobyte limit.

InfringingUrl

Field	Type	Description
url	string	A URL that infringes upon the Work. 8 kilobyte limit.

EntityNoticeRole

Field	Type	Description
name	string	The name of the role, one of "principal", "agent", "recipient", "sender", "target", "issuing_court", "plaintiff", "defendant" or "submitter". "recipient" and "sender" are displayed for all Notice Types on the Lumen website after automatic redaction.
entity_id	integer	(Optional) The ID of an existing entity. You should not specify the entity_attributes values below if you include entity_id here.
entity_attributes	Entity	A new entity that has this role on this notice. If you specify entity_attributes, you should not specify an entity_id

Note: If not explicitly provided, recipient entities will be assigned based on the user performing the submission (as identified by the authentication token used).

The submitter entity will always be assigned based on the user performing the submission (as identified by the authentication token used).

Entity

Field	Type	Description
name	string	Required
kind	string	one of "organization" or "individual"
address_line_1	string
address_line_2	string
city	string
state	string
zip	string
country_code	string	A two-digit ISO country code
phone	string
email	string
url	string

FileUpload

File uploading can only be used when submitting as form data. See the Submit Data section.

Field	Type	Description
kind	string	One of "original" or "supporting"
file	string	Path to a file on the client's system.

Language

The currently supported two-digit language codes are based on what Google Translate supports, and are currently:

iso	Country	iso	Country
af	Afrikaans	ar	Arabic
be	Belarusian	bg	Bulgarian
ca	Catalan; Valencian	cs	Czech
cy	Welsh	da	Danish
de	German	el	Greek, Modern
en	English	eo	Esperanto
es	Spanish; Castilian	et	Estonian
fa	Persian	fi	Finnish
fr	French	ga	Irish
gl	Galician	hi	Hindi
hr	Croatian	ht	Haitian; Haitian Creole
hu	Hungarian	id	Indonesian
is	Icelandic	it	Italian
iw	Hebrew	ja	Japanese
ko	Korean	lt	Lithuanian
lv	Latvian	mk	Macedonian
ml	Malayalam	ms	Malay
mt	Maltese	nl	Dutch
no	Norwegian	pl	Polish
pt	Portuguese	ro	Romanian
ru	Russian	si	Sinhala
sk	Slovak	sl	Slovene
sq	Albanian	sr	Serbian
sv	Swedish	sw	Swahili
th	Thai	tl	Tagalog
tr	Turkish	uk	Ukrainian
vi	Vietnamese	yi	Yiddish
yo	Yoruba	zh	Chinese

Submit Data

When creating a notice through the API, there are two ways to submit notice data: as form data or as JSON data. Each has advantages and disadvantages described below.

Data submission must include an authentication token (see Authentication).

As Form Data

Form data is useful for attaching files to the notice as the files can be included by name and location on a local drive.

This is the suggested method as it is more efficient at attaching files, which are transmitted as binary data.

Sample using curl

In curl, use the @ symbol to specify a local file.

curl -kv \
     -H "User-Agent: SomeUserAgentHere" \
     -H "X-Authentication-Token: abc123" \
     -F "notice[type]=Defamation" \
     -F "notice[title]=Defamation report via form data" \
     -F "notice[file_uploads_attributes][0][kind]=original" \
     -F "notice[file_uploads_attributes][0][file]=@Defamation complaint.docx" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://www.bad.com/people/296" \
     https://lumendatabase.org/notices

As JSON Data

JSON data is better if you do not have to submit file attachments as the syntax is simpler.

curl -H "User-Agent: SomeUserAgentHere" \
     -H "Accept: application/json" \
     -H "X-Authentication-Token: abc123" \
     -H "Content-type: application/json" \
     -d '{ "notice": { "type": "Defamation", "title": "Defamation report via JSON data" } }' \
     https://lumendatabase.org/notices

Example Requests

Example Request Using Entity Attributes

As Form Data

curl -kv \
     -H "User-Agent: SomeUserAgentHere"
     -H "X-Authentication-Token: abc123" \
     -F "notice[type]=DMCA" \
     -F "notice[title]=DMCA (Copyright) Complaint to Google" \
     -F "notice[subject]=Infringement Notfication via Blogger Complaint" \
     -F "notice[date_sent]=2014-07-04" \
     -F "notice[date_received]=2014-07-04" \
     -F "notice[source]=Online form" \
     -F "notice[body]=Victim of Infringement requests removal of content on Blogger." \
     -F "notice[topic_ids][]=19", \
     -F "notice[topic_ids][]=27", \
     -F "notice[tag_list]=movies, disney, youtube", \
     -F "notice[jurisdiction_list]=US, CA", \
     -F "notice[action_taken]=Partial", \
     -F "notice[file_uploads_attributes][0][kind]=original" \
     -F "notice[file_uploads_attributes][0][file]=@original_file.txt" \
     -F "notice[file_uploads_attributes][1][kind]=supporting" \
     -F "notice[file_uploads_attributes][1][file]=@supporting_file.txt" \
     -F "notice[works_attributes][0][description]=Lion King Video" \
     -F "notice[works_attributes][0][copyrighted_urls_attributes][0][url]=http://disney.com/lion_king.mp4" \
     -F "notice[works_attributes][0][copyrighted_urls_attributes][0][url]=http://disney.com/lion_king.mov" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_1" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_2" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_3" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_4" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_5" \
     -F "notice[entity_notice_roles_attributes][0][name]=recipient" \
     -F "notice[entity_notice_roles_attributes][0][entity_attributes][name]=Google" \
     -F "notice[entity_notice_roles_attributes][0][entity_attributes][kind]=organization" \
     -F "notice[entity_notice_roles_attributes][0][entity_attributes][address_line_1]=1600 Amphitheatre Parkway" \
     -F "notice[entity_notice_roles_attributes][0][entity_attributes][city]=Mountain View" \
     -F "notice[entity_notice_roles_attributes][0][entity_attributes][state]=CA" \
     -F "notice[entity_notice_roles_attributes][0][entity_attributes][zip]=94043" \
     -F "notice[entity_notice_roles_attributes][0][entity_attributes][country_code]=US" \
     -F "notice[entity_notice_roles_attributes][1][name]=sender" \
     -F "notice[entity_notice_roles_attributes][1][entity_attributes][name]=Joe Lawyer" \
     -F "notice[entity_notice_roles_attributes][1][entity_attributes][kind]=individual" \
     -F "notice[entity_notice_roles_attributes][1][entity_attributes][address_line_1]=1234 Anystreet St." \
     -F "notice[entity_notice_roles_attributes][1][entity_attributes][city]=Anytown" \
     -F "notice[entity_notice_roles_attributes][1][entity_attributes][state]=CA" \
     -F "notice[entity_notice_roles_attributes][1][entity_attributes][zip]=94044" \
     -F "notice[entity_notice_roles_attributes][1][entity_attributes][country_code]=US" \
     https://lumendatabase.org/notices

As JSON Data

curl -v \
  -H "User-Agent: SomeUserAgentHere" \
  -H "Accept: application/json" \
  -H "Content-type: application/json" \
  'https://lumendatabase.org/notices/' \
  -d '{
  "notice": {
    "title": "DMCA (Copyright) Complaint to Google",
    "type": "DMCA",
    "subject": "Infringement Notfication via Blogger Complaint",
    "date_sent": "2013-05-22",
    "date_received": "2013-05-23",
    "source": "Online form",
    "topic_ids": [19, 27],
    "tag_list": "movies, disney, youtube",
    "jurisdiction_list": "US, CA",
    "action_taken": "Partial",
    "works_attributes": [
      {
        "description": "Lion King Video",
        "copyrighted_urls_attributes": [
          { "url": "http://disney.com/lion_king.mp4" },
          { "url": "http://disney.com/lion_king.mov" }
        ],
        "infringing_urls_attributes": [
          { "url": "http://youtube.com/bad_url_1" },
          { "url": "http://youtube.com/bad_url_2" },
          { "url": "http://youtube.com/bad_url_3" },
          { "url": "http://youtube.com/bad_url_4" },
          { "url": "http://youtube.com/bad_url_5" }
        ]
      }
    ],
    "entity_notice_roles_attributes": [
      {
        "name": "recipient",
        "entity_attributes": {
          "name": "Google",
          "kind": "organization",
          "address_line_1": "1600 Amphitheatre Parkway",
          "city": "Mountain View",
          "state": "CA", 
          "zip": "94043",
          "country_code": "US"
        }
      },
      {
        "name": "sender",
        "entity_attributes": {
          "name": "Joe Lawyer",
          "kind": "individual",
          "address_line_1": "1234 Anystreet St.",
          "city": "Anytown",
          "state": "CA",
          "zip": "94044",
          "country_code": "US"
        }
      }
    ]
  }
}' 

Example Request Using Entity ID

As Form Data

curl -kv \
     -H "User-Agent: SomeUserAgentHere" \
     -H "X-Authentication-Token: abc123" \
     -F "notice[type]=DMCA" \
     -F "notice[title]=DMCA (Copyright) Complaint to Google" \
     -F "notice[subject]=Infringement Notfication via Blogger Complaint" \
     -F "notice[date_sent]=2014-07-04" \
     -F "notice[date_received]=2014-07-04" \
     -F "notice[source]=Online form" \
     -F "notice[body]=Victim of Infringement requests removal of content on Blogger." \
     -F "notice[topic_ids][]=19", \
     -F "notice[topic_ids][]=27", \
     -F "notice[tag_list]=movies, disney, youtube", \
     -F "notice[jurisdiction_list]=US, CA", \
     -F "notice[action_taken]=Partial", \
     -F "notice[file_uploads_attributes][0][kind]=original" \
     -F "notice[file_uploads_attributes][0][file]=@original_file.txt" \
     -F "notice[file_uploads_attributes][1][kind]=supporting" \
     -F "notice[file_uploads_attributes][1][file]=@supporting_file.txt" \
     -F "notice[works_attributes][0][description]=Lion King Video" \
     -F "notice[works_attributes][0][copyrighted_urls_attributes][0][url]=http://disney.com/lion_king.mp4" \
     -F "notice[works_attributes][0][copyrighted_urls_attributes][0][url]=http://disney.com/lion_king.mov" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_1" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_2" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_3" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_4" \
     -F "notice[works_attributes][0][infringing_urls_attributes][0][url]=http://youtube.com/bad_url_5" \
     -F "notice[entity_id]=1" \
     https://lumendatabase.org/notices

As JSON data

curl -v \
  -H "User-Agent: SomeUserAgentHere" \
  -H "Accept: application/json" \
  -H "Content-type: application/json" \
  -H "X-Authentication-Token: abc123" \
  'https://lumendatabase.org/notices/' \
  -d '{
  "notice": {
    "title": "DMCA (Copyright) Complaint to Google",
    "type": "DMCA",
    "subject": "Infringement Notfication via Blogger Complaint",
    "date_sent": "2013-05-22",
    "date_received": "2013-05-23",
    "source": "Online form",
    "topic_ids": [19, 27],
    "tag_list": "movies, disney, youtube",
    "jurisdiction_list": "US, CA",
    "action_taken": "Partial",
    "works_attributes": [
      {
        "description": "Lion King Video",
        "copyrighted_urls_attributes": [
          { "url": "http://disney.com/lion_king.mp4" },
          { "url": "http://disney.com/lion_king.mov" }
        ],
        "infringing_urls_attributes": [
          { "url": "http://youtube.com/bad_url_1" },
          { "url": "http://youtube.com/bad_url_2" },
          { "url": "http://youtube.com/bad_url_3" },
          { "url": "http://youtube.com/bad_url_4" },
          { "url": "http://youtube.com/bad_url_5" }
        ]
      }
    ],
    "entity_id": 1
  }
}' 

Example Responses

Successful Responses

Successful responses will have HTTP status 201 (Created) and include a Location: HTTP header with the location of the created object.

Note: Unimportant headers have been removed.

HTTP/1.1 201 Created 
Location: http://localhost:3000/notices/4
Content-Type: application/json; charset=utf-8
Cache-Control: max-age=0, private, must-revalidate
Date: Wed, 05 Jun 2013 20:43:34 GMT
Content-Length: 1

Unsuccessful Responses

Most unsuccessful responses will have HTTP status 422 (Unprocessable Entity) and include a JSON response enumerating the validation failures.

If requests are too fast or too numerous, various other HTTP error codes may appear. If these are persistent, feel free to contact us and we will look into it. Be sure to include your IP address, user agent, and API key, if any.

{
  "works":["can't be blank"],
  "entity_notice_roles":["can't be blank"],
  "title":["can't be blank"]
}

Notice Type Mapping

All notices are added to the system through the API using the same named parameters (see Create Notice). However, fields are "mapped" depending on the Notice Type when serialized as JSON through a search request or when requested individually.

Counternotice

Attributes remain unchanged from ingestion.

CourtOrder

Ingestion Field Name	Mapped Field Name	Removed?
body	explanation
regulations	laws_referenced
works.description	works.subject_of_court_order
works.infringing_urls	works.targetted_urls
works.copyrighted_urls		Yes

DataProtection

Ingestion Field Name	Mapped Field Name	Removed?
body	legal_complaint
works.description	works.complaint
works.infringing_urls	works.urls_mentioned_in_request
works.copyrighted_urls		Yes

Defamation

Ingestion Field Name	Mapped Field Name	Removed?
body	legal_complaint
works.infringing_urls	works.defamatory_urls
works.copyrighted_urls		Yes

DMCA

Attributes remain unchanged from ingestion.

GovernmentRequest

Ingestion Field Name	Mapped Field Name	Removed?
body	explanation
works.description	works.subject
works.infringing_urls	works.urls_of_original_work
works.copyrighted_urls	works.urls_mentioned_in_request

LawEnforcementRequest

Ingestion Field Name	Mapped Field Name	Removed?
body	explanation
works.description	works.subject_of_enforcement_request
works.infringing_urls	works.urls_in_request
works.copyrighted_urls	works.original_work_urls
regulation_list	regulations

Other

Ingestion Field Name	Mapped Field Name	Removed?
body	explanation
works.description	works.complaint
works.infringing_urls	works.problematic_urls
works.copyrighted_urls	works.original_work_urls

PrivateInformation

Ingestion Field Name	Mapped Field Name	Removed?
body	explanation
works.description	works.complaint
works.infringing_urls	works.urls_with_private_information
works.copyrighted_urls		Yes

Trademark

Ingestion Field Name	Mapped Field Name	Removed?
mark_registration_number
works.description	marks.description
works.infringing_urls	marks.infringing_urls
works.copyrighted_urls		Yes

Epoch format

In some of the facets we use the Unix epoch time format (https://en.wikipedia.org/wiki/Unix_time) in milliseconds.

An easy way to convert time from a human-readable format to epoch is using the https://www.epochconverter.com website. Use the converter to convert it to milliseconds.

Tuesday, 1 January 2019 12:00:00 = 1546344000000 milliseconds.

Scripting

Lumen Database does not offer packaged downloads of data but we also do not forbid researchers from writing scripts to perform searches or retrieve notice data. However, we do have a few suggestions that will help keep our system responsive and also help us if you run into any issues.

1. Either request .json endpoints or include the "Accept: application/json" header.

2. Consider adding an accept-encoding header to indicate you prefer compressed results (in curl, -H "Accept-Encoding: gzip").

3. Make sure you are using your authentication token to avoid rate limiting.

Understanding the Search Page Format

URL Formats

A search for notices including the term "Lumen" looks like this: https://www.lumendatabase.org/notices/search? &term=Lumen.

For a search that has multiple words, &term-require-all=true is a handy modifier: lumendatabase.org/notices/search?term=Lumen+Project &term-require-all=true.

To request the search results page in .json format, add the .json extension to the URL like so: lumendatabase.org/notices/search .json ?term=Lumen+Project&term-require-all=true.

And then to get other search result pages, simply add the &page= modifier: lumendatabase.org/notices/search.json?sort_by=&term-require-all=true&term=Lumen+Project&utf8=%E2%9C%93 &page=2.