Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add table rdap_domain (#36) #46

Merged
merged 9 commits into from
Jan 18, 2024
Merged

Add table rdap_domain (#36) #46

merged 9 commits into from
Jan 18, 2024

Conversation

ParthaI
Copy link
Contributor

@ParthaI ParthaI commented Oct 27, 2023

Example query results

Results 
Add example SQL query results here (please include the input queries as well)

@lyda @ParthaI
First pass at supporting RDAP (#36)
6c52aba 
Co-authored-by: ParthaI <47887552+ParthaI@users.noreply.github.com>
@ParthaI ParthaI self-assigned this Oct 27, 2023
@bigdatasourav bigdatasourav linked an issue Nov 6, 2023 that may be closed by this pull request
Whois deprecation - migrate to RDAP #35
Closed
cbruno10
cbruno10 requested changes Nov 6, 2023
View reviewed changes
Copy link
Contributor

@cbruno10 cbruno10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ParthaI Please see comment, thanks!

whois/table_rdap_domain.go Outdated
{Name: "domain_id", Type: proto.ColumnType_STRING, Transform: transform.FromField("Handle").NullIfZero(), Description: "Unique identifier for the domain."},
{Name: "object_class_name", Type: proto.ColumnType_STRING, Description: "String 'domain' representing the object type in RDAP."},
{Name: "ldh_name", Type: proto.ColumnType_STRING, Description: "Textual representation of DNS names.", Transform: transform.FromField("LDHName")},
{Name: "port43", Type: proto.ColumnType_STRING, Description: "Hostname of Registry WHOIS server."},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description seems a bit strange based on the column name, do you have an example of the data we get back from the SDK for this field/column?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cbruno10, I queried this table to retrieve the 'port43' value by using multiple domain names. However, in all cases, we received null or empty data.

I found the description mentioned in the document, so I included it here.

> select domain, port43 from rdap_domain where domain = 'example.com'
+-------------+--------+
| domain      | port43 |
+-------------+--------+
| example.com |        |
+-------------+--------+

Time: 3.2s. Rows fetched: 1. Hydrate calls: 0.
> select domain, port43 from rdap_domain where domain = 'google.org'
+------------+--------+
| domain     | port43 |
+------------+--------+
| google.org |        |
+------------+--------+

Time: 4.5s. Rows fetched: 1. Hydrate calls: 0.
> select domain, port43 from rdap_domain where domain = 'microsoft.info'
+----------------+--------+
| domain         | port43 |
+----------------+--------+
| microsoft.info |        |
+----------------+--------+

Time: 2.2s. Rows fetched: 1. Hydrate calls: 0.
> select domain, port43 from rdap_domain where domain = 'whitehouse.gov'
+--------+--------+
| domain | port43 |
+--------+--------+
+--------+--------+

Time: 354ms.
> select domain, port43 from rdap_domain where domain = 'amazon.in'
+--------+--------+
| domain | port43 |
+--------+--------+
+--------+--------+

Time: 399ms.
> select domain, port43 from rdap_domain where domain = 'amazon.com'
+------------+--------+
| domain     | port43 |
+------------+--------+
| amazon.com |        |
+------------+--------+

Time: 1.2s. Rows fetched: 1. Hydrate calls: 0.

cbruno10
cbruno10 reviewed Jan 4, 2024
View reviewed changes
Copy link
Contributor

@cbruno10 cbruno10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ParthaI Please see comments, thanks!

whois/table_rdap_domain.go Outdated
// Top columns
{Name: "domain", Type: proto.ColumnType_STRING, Transform: transform.FromQual("domain"), Description: "Domain name the RDAP information relates to."},

{Name: "domain_id", Type: proto.ColumnType_STRING, Transform: transform.FromField("Handle").NullIfZero(), Description: "Unique identifier for the domain."},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the column be called handle, if the field name is Handle? Is there a reason you chose domain_id instead? Also, where does the description come from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cbruno10, I've renamed the column domain_id to handle to ensure consistency with the API response. For most column descriptions, I referred to this documentation. For those not covered in the documentation, I searched online and selected descriptions that best matched the data contained in each column.

https://datatracker.ietf.org/doc/html/rfc7483.html#section-5.1
https://datatracker.ietf.org/doc/html/rfc7483#section-5.3

docs/tables/rdap_domain.md Outdated
@@ -0,0 +1,194 @@
# Table: rdap_domain
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ParthaI Can you please update this table doc so it matches our latest doc format (if it doesn't already)?

Copy link
Contributor Author

@ParthaI ParthaI Jan 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cbruno10, I have updated the doc to match our latest doc format.

cbruno10
cbruno10 requested changes Jan 17, 2024
View reviewed changes
Copy link
Contributor

@cbruno10 cbruno10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ParthaI Please see review comments, thanks!

docs/tables/rdap_domain.md Outdated
@@ -0,0 +1,374 @@
---
title: "Steampipe Table: rdap_domain - Query Whois Domains using SQL"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
title: "Steampipe Table: rdap_domain - Query Whois Domains using SQL"
title: "Steampipe Table: rdap_domain - Query RDAP Domains using SQL"

docs/tables/rdap_domain.md Outdated

## Examples

### Basic whois info
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Basic whois info
### Basic RDAP info

whois/plugin.go
@@ -13,6 +13,7 @@ func Plugin(ctx context.Context) *plugin.Plugin {
DefaultTransform: transform.FromGo().NullIfZero(),
TableMap: map[string]*plugin.Table{
"whois_domain": tableWhoisDomain(ctx),
"rdap_domain": tableRdapDomain(ctx),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move above so we keep alphabetical order

@bigdatasourav bigdatasourav changed the title First pass at supporting RDAP (#36) Add table rdap_domain (#36) Jan 18, 2024
@bigdatasourav bigdatasourav merged commit 6bff8dc into main Jan 18, 2024
1 check passed
@bigdatasourav bigdatasourav deleted the add-support-for-rdpa branch January 18, 2024 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cbruno10 cbruno10 cbruno10 approved these changes

Assignees

@ParthaI ParthaI

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Whois deprecation - migrate to RDAP
4 participants
@ParthaI @cbruno10 @bigdatasourav @lyda