Search Resource

This endpoint allows you to search the DarkOwl Vision database and receive detailed results.

The 'q' field is the primary search field recommended for use and determines the relevancy of the results that are provided. Additionally, if the 'highlight' parameter is set to true, data in this parameter will be highlighted in the body field of the results.

We recommend using filters whenever possible for better performance and results. As filters are not used in the calculation of relevancy, you may wish to use both the filter and the 'q' parameter in some cases. For example: when searching for an email address such as `joe.user@email.com` using the email filter, including this value in the 'q' field will improve results, return in a more timely fashion, and highlight the term in the result (if requested).

GET /api/v1/search

Request Parameters
name type description default constraints multivalued
cccn_max query Filter. Credit card count (upper bound). Use to filter to results containing at most this number of credit cards. Can be used in conjunction with 'cccn_min' to form a range.   max: 999999, min: 0 no
cccn_min query Filter. Credit card count (lower bound). Use to filter to results containing at least this number of credit cards. Can be used in conjunction with 'cccn_max' to form a range.   max: 999999, min: 0 no
ccn query Filter. Use to search for a desired credit card number in the body of a result. Input is 6-20 digits and supports trailing wildcards. Use in conjunction with the ‘q’ parameter and highlight=true to highlight the number in the body of a result.     yes
cemail_max query Filter. Email count (upper bound). Use to filter to results containing at most this number of credit cards. Can be used in conjunction with 'cemail_min' to form a range.   max: 999999, min: 0 no
cemail_min query Filter. Email count (lower bound). Use to filter to results containing at least this number of credit cards. Can be used in conjunction with 'cemail_max' to form a range.   max: 999999, min: 0 no
count query Result Option. Allows you to set the number of results returned per request. The default is 20 results, which is the maximum number of results allowed per request. We recommend using this parameter while developing and for debugging purposes. 20 max: 20, min: 1 no
cssn_max query Filter. Social security number count (upper bound). Use to filter to results containing at most this number of social security numbers. Can be used in conjunction with 'cssn_min' to form a range.   max: 999999, min: 0 no
cssn_min query Filter. Social security number count (lower bound). Use to filter to results containing at least this number of social security numbers. Can be used in conjunction with 'cssn_max' to form a range.   max: 999999, min: 0 no
detail query Result Option. Allows you to return full detail about a result, or selected metadata. Options include:
  • full: returns all fields associated with a result, including the document body
  • nonbody: returns all fields except the document body
  • snippet: returns all fields, plus a portion (snippet) of the document body, around a highlighted term. When selecting this option, the 'highlight' parameter is automatically included as part of the request.
full regex: ^(full|snippet|nonbody)$ no
domain query Filter. Use to filter to results collected from one or more domains. Exclude domains by prefixing with a hyphen (-).     yes
email query Filter. Use to search for an individual email address. Use in conjunction with the ‘q’ parameter and highlight=true to highlight the email in the body of a result. Note: the 'emailDomain' parameter is used to find email addresses within an entire domain.     yes
emailDomain query Filter. Use to search for all email addresses associated with a domain, which are found in the body of a result. Input format is domain.com or subdomain.domain.com, without special characters such as the @ symbol.     yes
from query Filter. Crawl date (lower bound). Use to filter to results collected before this datetime (UTC). Input format is in Zulu: YYYY-MM-DDThh:mm:ssZ. Can be used in conjunction with the 'to' parameter to form a range.     no
hack_max query Filter. Hackishness (upper bound). Use to filter to results containing at most this hackishness rating. Input is a decimal between 0.0 and 1.0. Can be used in conjunction with 'hack_min' to form a range.   max: 1.0, min: 0.0 no
hack_min query Filter. Hackishness (lower bound). Use to filter to results containing at least this hackishness rating. Input is a decimal between 0.0 and 1.0. Can be used in conjunction with 'hack_max' to form a range.   max: 1.0, min: 0.0 no
has query Filter. Use to filter to results that contain a certain attribute. Options include:
  • ccn: the result must contain at least one credit card number
  • ssn: the result must contain at least one email address
  • email: the result must contain at least one social security number
    yes
highlight query Result Option. If set to true, allows you to return highlighted values sent in the ‘q’ parameter in the body field. false boolean no
ip query Filter. Use to search for an ip address in the body of a result. Use in conjunction with the ‘q’ parameter and highlight=true to highlight the email in the body of a result.     yes
lang query Filter. Use to filter to results in desired languages. Language is assigned by DarkOwl Vision at the time of ingestion. A list of language values is available at darkowl.com/docs/vision-resources. Exclude languages by prefixing with a hyphen (-).     yes
loc query Filter. Use to filter to results registered in certain countries. Country is determined by a geolocation lookup on the target ip (if available). A list of country values is available at darkowl.com/docs/vision-resources. Exclude countries by prefixing with a hyphen (-).     yes
offset query Result Option. Allows you to skip a number of results; this is used for pagination. For example (using the default count of 20): offset=0 will return the first 20 results, offset=20 will return the second 20 results, offset=40 will return the third 20 results, and so on. 0 max: 380, min: 0 no
q query Base Search. The primary search field recommended for use, and determines the relevancy of the results that are provided. AND, OR, NOT is supported in this parameter. Use quotations around phrases, and parentheses to form subqueries. Exclude items with NOT or a hyphen (-). Special characters used in this field must be properly escaped. Wildcards are generally allowed, except for leading wildcards. Use highlight=true to return results with the data sent in this parameter highlighted in the body field.     no
req query Result Option. Allows you to return the request data used to run the search, along with the results. We recommend using this parameter while developing and for debugging purposes.   boolean no
similar query Result Option. Allows you to return all results or remove results that are similar, per request. Note that setting this to false may return fewer documents than the count selected. true boolean no
sort query Result Option. Allows you to return the results in a particular order. Options include:
  • r: Relevancy
  • h: Hackishness
  • d: Date Crawled (most recent)
r regex: ^[hrd]$ no
ssn query Filter. Use to search for a desired social security numbers in the body of a result. Input format must include hyphens: NNN-NN-NNNN. Use in conjunction with the ‘q’ parameter and highlight=true to highlight the number in the body of a result.     yes
to query Filter. Crawl date (upper bound). Use to filter to results collected before this datetime (UTC). Input format is in Zulu: YYYY-MM-DDThh:mm:ssZ. Can be used in conjunction with the 'from' parameter to form a range.     no
Response Body
media type data type
application/json SearchResults (JSON)

Example

Request
GET /api/v1/search
Accept: application/json

              
Response
HTTP/1.1 200 OK
Content-Type: application/json

        
        {
  "request" : {
    "q" : "...",
    "loc" : [ "...", "..." ],
    "lang" : [ "...", "..." ],
    "topic" : [ "...", "..." ],
    "domain" : [ "...", "..." ],
    "from" : "...",
    "to" : "...",
    "hack_min" : 12345.0,
    "hack_max" : 12345.0,
    "has" : [ "...", "..." ],
    "ccn" : [ "...", "..." ],
    "ssn" : [ "...", "..." ],
    "email" : [ "...", "..." ],
    "emailDomain" : [ "...", "..." ],
    "ip" : [ "...", "..." ],
    "cccn_min" : 12345,
    "cccn_max" : 12345,
    "cssn_min" : 12345,
    "cssn_max" : 12345,
    "cemail_min" : 12345,
    "cemail_max" : 12345
  },
  "total" : 12345,
  "resultCount" : 12345,
  "results" : [ {
    "id" : "...",
    "url" : "...",
    "title" : "...",
    "body" : "...",
    "snippet" : "...",
    "crawlDate" : "...",
    "hackishness" : 12345.0,
    "ip" : "...",
    "headers" : [ "...", "..." ],
    "country" : "...",
    "fileSize" : 12345,
    "domain" : "...",
    "ccns" : [ "...", "..." ],
    "ssns" : [ "...", "..." ],
    "emails" : [ "...", "..." ]
  }, {
    "id" : "...",
    "url" : "...",
    "title" : "...",
    "body" : "...",
    "snippet" : "...",
    "crawlDate" : "...",
    "hackishness" : 12345.0,
    "ip" : "...",
    "headers" : [ "...", "..." ],
    "country" : "...",
    "fileSize" : 12345,
    "domain" : "...",
    "ccns" : [ "...", "..." ],
    "ssns" : [ "...", "..." ],
    "emails" : [ "...", "..." ]
  } ]
}