Парсинг данных JSON WHOIS от ARIN
Я работаю над тем, как "автоматизировать" процесс проверки журналов, и часть (предполагаемого) процесса включает в себя поиск в WHOIS неизвестных IP-адресов. Я выяснил, как получить информацию из ARIN в формате JSON, и я почти получил весь объект, созданный и заполненный сейчас. Тем не менее, я столкнулся с проблемой с одним конкретным вложенным объектом - vcardArray объект под entities,
Я использую библиотеку Newsonoft Json.NET для десериализации объекта (ов) в ответе JSON с сервера ARIN. Большая часть информации будет десериализована правильно, но раздел, который я не могу правильно проанализировать, таков:
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "fn", { }, "text", "Cox Communications Inc." ], [ "adr", {
"label" : "1400 Lake Hearn Dr.\nAtlanta\nGA\n30319\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "kind", { }, "text", "org" ] ] ]
Когда я закомментирую vcardArray собственность в моем Entity объект, остальная часть текста JSON анализируется без ошибок. Но, поскольку я не знаю, как структурировать свойства vCard объект, он выдаст исключение, если я попытаюсь использовать его. Я пытался определить это просто как List(Of String), но это не удается, когда он пытается проанализировать "подмассивы" в JSON. Я даже пытался сделать это List(Of List(Of String)), но затем он выдает исключение, когда пытается проанализировать значения одной строки.
Я (очевидно) еще не проделал большую работу с JSON, поэтому я не уверен, как мне структурировать vCard объект для учета отсутствия фактических пар ключ / значение и вложенных объектов массива в этом разделе. Любая помощь будет принята с благодарностью.
Код VB.NET:
Imports System.Net
Imports System.IO
Public Class WHOIS
Private SearchIP As IPAddress
#Region "JSON CLASS OBJECTS"
Public Class ARINDetail
Public Property rdapConformance As List(Of String)
Public Property notices As List(Of Notice)
Public Property handle As String
Public Property startAddress As String
Public Property endAddress As String
Public Property ipVersion As String
Public Property name As String
Public Property type As String
Public Property country As String
Public Property parentHandle As String
Public Property objectClassName As String
Public Property entities As List(Of Entity)
Public Property events As List(Of [Event])
Public Property remarks As List(Of Remark)
Public Property links As List(Of Link)
End Class
Public Class Notice
Public Property title As String
Public Property description As List(Of String)
Public Property links As List(Of Link)
End Class
Public Class Entity
Public Property handle As String
Public Property vcardArray As List(Of vCard)
Public Property roles As List(Of String)
Public Property remarks As List(Of Remark)
Public Property objectClassName As String
Public Property links As List(Of Link)
Public Property events As List(Of [Event])
Public Property entities As List(Of Entity)
End Class
Public Class [Event]
Public Property eventAction As String
Public Property eventdate As String
End Class
Public Class Remark
Public Property title As String
Public Property description As List(Of String)
End Class
Public Class vCard
' ** THIS IS THE CLASS I CAN'T FIGURE OUT HOW TO CONSTRUCT
End Class
Public Class Link
Public Property value As String
Public Property rel As String
Public Property href As String
Public Property type As String
End Class
#End Region
#Region "CONSTRUCTORS"
Public Sub New(ByVal IP As String)
Try
SearchIP = IPAddress.Parse(IP)
Catch ex As Exception
Throw New ArgumentException("The IP address provided is invalid")
End Try
End Sub
Public Sub New(ByVal IP As IPAddress)
SearchIP = IP
End Sub
#End Region
#Region "PUBLIC METHODS"
Public Function GetWHOISData() As String
Dim ARINJSON As String = String.Empty
ARINJSON = RequestARINData()
Dim Details As ARINDetail = Newtonsoft.Json.JsonConvert.DeserializeObject(Of ARINDetail)(ARINJSON)
End Function
#End Region
#Region "PRIVATE METHODS"
Private Function RequestARINData() As String
Dim RequestURL As String = "https://rdap.arin.net/registry/ip/" & Me.SearchIP.ToString
Dim JSONResponse As String = String.Empty
Dim HTTPResponse As System.Net.WebResponse
Dim HTTPRequest As System.Net.WebRequest = HttpWebRequest.Create(RequestURL)
HTTPResponse = HTTPRequest.GetResponse
Using sr As StreamReader = New StreamReader(HTTPResponse.GetResponseStream)
JSONResponse = sr.ReadToEnd
sr.Close()
End Using
Return JSONResponse
End Function
#End Region
End Class
Полный ответ JSON от ARIN
{
"rdapConformance" : [ "rdap_level_0" ],
"notices" : [ {
"title" : "Terms of Service",
"description" : [ "By using the ARIN RDAP/Whois service, you are agreeing to the RDAP/Whois Terms of Use" ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "about",
"type" : "text/html",
"href" : "https://www.arin.net/whois_tou.html"
} ]
} ],
"handle" : "NET-68-15-96-0-1",
"startAddress" : "068.015.096.000",
"endAddress" : "068.015.127.255",
"ipVersion" : "v4",
"name" : "NETBLK-OK-CBS-68-15-96-0",
"parentHandle" : "NET-68-0-0-0-1",
"remarks" : [ {
"title" : "Registration Comments",
"description" : [ "For legal requests/assistance please use the following contact information:", "", "Cox Subpoena Phone: 404-269-0100", "", "Cox Subpoena Info: http://www.cox.com/policy/leainformation/default.asp" ]
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2007-03-27T13:14:28-04:00"
}, {
"eventAction" : "registration",
"eventDate" : "2002-04-21T00:00:00-04:00"
} ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/ip/068.015.096.000"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/net/NET-68-15-96-0-1"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "up",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/ip/068.000.000.000/12"
} ],
"entities" : [ {
"handle" : "CXA",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "fn", { }, "text", "Cox Communications Inc." ], [ "adr", {
"label" : "1400 Lake Hearn Dr.\nAtlanta\nGA\n30319\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "kind", { }, "text", "org" ] ] ],
"roles" : [ "registrant" ],
"remarks" : [ {
"title" : "Registration Comments",
"description" : [ "For legal requests/assistance please use the\r", "following contact information:\r", "Cox Subpoena Info: https://www.cox.com/aboutus/policies/law-enforcement-and-subpoenas-information.html" ]
} ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/CXA"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/org/CXA"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2017-05-30T09:52:56-04:00"
} ],
"entities" : [ {
"handle" : "BERUB3-ARIN",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "adr", {
"label" : "6305-A Peachtree Dunwoody Road,\nAtlanta\nGA\n30328\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "fn", { }, "text", "Tori Berube" ], [ "n", { }, "text", [ "Berube", "Tori", "", "", "" ] ], [ "kind", { }, "text", "individual" ], [ "email", { }, "text", "tori.berube@cox.com" ], [ "tel", {
"type" : [ "work", "voice" ]
}, "text", "+1-404-269-4416" ] ] ],
"roles" : [ "administrative", "technical" ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/BERUB3-ARIN"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/poc/BERUB3-ARIN"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2017-04-04T08:47:01-04:00"
}, {
"eventAction" : "registration",
"eventDate" : "2014-03-18T13:16:56-04:00"
} ],
"status" : [ "validated" ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
}, {
"handle" : "MEROL3-ARIN",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "adr", {
"label" : "6305-A Peachtree Dunwoody Rd\nAtlanta\nGA\n30328\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "fn", { }, "text", "Cari Merola" ], [ "n", { }, "text", [ "Merola", "Cari", "", "", "" ] ], [ "kind", { }, "text", "individual" ], [ "email", { }, "text", "cari.merola@cox.com" ], [ "tel", {
"type" : [ "work", "voice" ]
}, "text", "+1-404-269-4416" ] ] ],
"roles" : [ "technical" ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/MEROL3-ARIN"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/poc/MEROL3-ARIN"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2017-05-10T10:37:30-04:00"
}, {
"eventAction" : "registration",
"eventDate" : "2017-05-10T10:37:30-04:00"
} ],
"status" : [ "validated" ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
}, {
"handle" : "ADA131-ARIN",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "adr", {
"label" : "6305 Peachtree dunwoody\nAtlanta\nGA\n30328\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "fn", { }, "text", "Alvin Demond Anderson" ], [ "n", { }, "text", [ "Anderson", "Alvin", "Demond", "", "" ] ], [ "kind", { }, "text", "individual" ], [ "email", { }, "text", "alvin.anderson@cox.com" ], [ "tel", {
"type" : [ "work", "voice" ]
}, "text", "+1-404-269-4416" ] ] ],
"roles" : [ "technical" ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/ADA131-ARIN"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/poc/ADA131-ARIN"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2016-08-30T17:24:55-04:00"
}, {
"eventAction" : "registration",
"eventDate" : "2014-09-09T13:19:41-04:00"
} ],
"status" : [ "validated" ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
}, {
"handle" : "RWA196-ARIN",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "adr", {
"label" : "1400 Lake Hearn Dr.\nAtlanta\nGA\n30319\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "fn", { }, "text", "Roderick Waldron" ], [ "n", { }, "text", [ "Waldron", "Roderick", "", "", "" ] ], [ "kind", { }, "text", "individual" ], [ "email", { }, "text", "abuse@cox.net" ], [ "email", { }, "text", "Roderick.Waldron@cox.com" ], [ "tel", {
"type" : [ "work", "voice" ]
}, "text", "+1-404-269-7626" ] ] ],
"roles" : [ "technical" ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/RWA196-ARIN"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/poc/RWA196-ARIN"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2016-11-22T12:32:08-05:00"
}, {
"eventAction" : "registration",
"eventDate" : "2009-04-24T10:54:11-04:00"
} ],
"status" : [ "validated" ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
}, {
"handle" : "NIA16-ARIN",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "adr", {
"label" : "6305-A Peachtree Dunwoody\nAtlanta\nGA\n30328\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "fn", { }, "text", "National IP Administrator" ], [ "org", { }, "text", "National IP Administrator" ], [ "kind", { }, "text", "group" ], [ "email", { }, "text", "tiffany.coleman@cox.com" ], [ "tel", {
"type" : [ "work", "voice" ]
}, "text", "+1-404-269-4416" ] ] ],
"roles" : [ "technical" ],
"remarks" : [ {
"title" : "Unvalidated POC",
"description" : [ "ARIN has attempted to validate the data for this POC, but has received no response from the POC since 2015-09-09" ]
} ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/NIA16-ARIN"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/poc/NIA16-ARIN"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2014-09-09T13:19:45-04:00"
}, {
"eventAction" : "registration",
"eventDate" : "2014-09-09T13:19:45-04:00"
} ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
}, {
"handle" : "BAABO-ARIN",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "adr", {
"label" : "1400 Lake Hearn Drive\nAtlanta\nGA\n30342\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "fn", { }, "text", "Aboubakr BA" ], [ "n", { }, "text", [ "BA", "Aboubakr", "", "", "" ] ], [ "kind", { }, "text", "individual" ], [ "email", { }, "text", "abuse@cox.net" ], [ "tel", {
"type" : [ "work", "voice" ]
}, "text", "+1-404-269-4416" ] ] ],
"roles" : [ "technical" ],
"remarks" : [ {
"title" : "Unvalidated POC",
"description" : [ "ARIN has attempted to validate the data for this POC, but has received no response from the POC since 2016-03-29" ]
} ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/BAABO-ARIN"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/poc/BAABO-ARIN"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2015-03-30T11:27:55-04:00"
}, {
"eventAction" : "registration",
"eventDate" : "2012-09-27T12:04:18-04:00"
} ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
}, {
"handle" : "IC146-ARIN",
"vcardArray" : [ "vcard", [ [ "version", { }, "text", "4.0" ], [ "adr", {
"label" : "1400 Lake Hearn Drive\nAtlanta\nGA\n30319\nUNITED STATES"
}, "text", [ "", "", "", "", "", "", "" ] ], [ "fn", { }, "text", "Cox Communications Inc" ], [ "org", { }, "text", "Cox Communications Inc" ], [ "kind", { }, "text", "group" ], [ "email", { }, "text", "abuse@cox.net" ], [ "tel", {
"type" : [ "work", "voice" ]
}, "text", "+1-404-269-7626" ] ] ],
"roles" : [ "abuse" ],
"remarks" : [ {
"title" : "Unvalidated POC",
"description" : [ "ARIN has attempted to validate the data for this POC, but has received no response from the POC since 2016-11-19" ]
} ],
"links" : [ {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "self",
"type" : "application/rdap+json",
"href" : "https://rdap.arin.net/registry/entity/IC146-ARIN"
}, {
"value" : "https://rdap.arin.net/registry/ip/68.15.102.5",
"rel" : "alternate",
"type" : "application/xml",
"href" : "https://whois.arin.net/rest/poc/IC146-ARIN"
} ],
"events" : [ {
"eventAction" : "last changed",
"eventDate" : "2015-11-20T08:52:56-05:00"
}, {
"eventAction" : "registration",
"eventDate" : "2002-02-12T11:20:45-05:00"
} ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
} ],
"port43" : "whois.arin.net",
"objectClassName" : "entity"
} ],
"port43" : "whois.arin.net",
"objectClassName" : "ip network"
}
РЕДАКТИРОВАТЬ: Я уехал в отпуск на неделю без подключения к Интернету, поэтому у меня не было возможности изучить это очень много, так как я разместил вопрос. Теперь, когда я вернулся с (несколько) ясной головой, я начал проводить дополнительные исследования именно того, чего я пытаюсь достичь.
Благодаря моему свободному использованию Google-fu я натолкнулся на этот вопрос, который, похоже, дает аналогичный ответ на то, что @dbc разместил в комментариях ниже. Однако я также обнаружил, что у vCards действительно есть JSON-эквивалент - jCard - как определено в RFC 7095 (выпущен в январе 2014 года), что заставляет меня поверить, что определенно должен быть более простой способ их десериализации. Очевидно, что JSON, который вызывает у меня проблемы, выглядит как массив этих объектов jCard, поэтому мне просто нужно найти "стандартный" способ десериализации jCard, чтобы я мог добавить его туда. Тогда я должен быть в состоянии просто объявить vcardArray объект как List(Of jCard)и у меня все готово.
Я делаю некоторые обновления в своей системе разработки прямо сейчас, поэтому мне придется более внимательно посмотреть, когда это будет сделано, но мне кажется, что библиотека Newtonsoft JSON должна иметь jCard объект сейчас.
Я все еще исследую, и я опубликую другое редактирование, когда у меня будет больше информации, но я решил опубликовать это и посмотреть, есть ли у кого-нибудь какие-либо мысли или другие советы, основанные на том, что я нашел до сих пор.