Ошибка H2O при вызове as.factor на фрейме данных H2O
Когда я называю следующую воспроизводимую doce:
install.packages("h2o", type = "source", repos =
"http://h2o-release.s3.amazonaws.com/h2o/rel-ueno/8/R")
library(rsparkling)
library(h2o)
library(sparklyr)
library(dplyr)
library(DBI)
library(readr)
library(stringr)
conf <- spark_config()
conf$'sparklyr.shell.executor-memory' <- "460G"
conf$'sparklyr.shell.driver-memory' <- "460G"
conf$spark.executor.cores <- 32
conf$spark.executor.memory <- "460G"
conf$spark.executor.instances <- 8
conf$spark.dynamicAllocation.enabled <- "false"
conf$maximizeResourceAllocation <- "true"
conf$spark.default.parallelism <- 32
sc <- spark_connect(master = "local", config = conf, version = "2.0.2")
h2o_context(sc ,strict_version_check = T)
h2o.init(nthreads = -1)
NFLPlayers <- c('Von Miller','Christian McCaffrey','Joe Thomas')
salary <- c(21000, 23400, 26800)
Valuable = c(1,1,1)
Rdata <- data.frame(NFLPlayers, salary, Valuable)
system.time(Test <- copy_to(sc, Rdata, "Rdata"))
Test1 = as_h2o_frame(sc, Test, strict_version_check = FALSE)
Test1$Valuable<-as.factor(Test1$Valuable)
На кадре данных H2O я получаю следующую ошибку.
ERROR: Unexpected HTTP Status code: 500 Server Error (url = http://127.0.0.1:54321/99/Rapids)
Error: lexical error: invalid char in json text.
<html> <head> <meta http-equiv=
(right here) ------^
Это совершенно новое, и это происходит не только на моем компьютере, но и на компьютере коллеги. Я перезапустил кластер. Переустановил sparklyr и Rsparkling. Я также изменил версии и постоянно получаю ту же ошибку. Есть идеи?
Некоторые из журналов:
06-19 22:46:24.991 127.0.0.1:54321 2158 #2868-244 INFO: POST /99/Rapids, parms: {ast=(tmp= RTMP_sid_8db5_1 (:= frame_rdd_58_b5fd354222706e76381948c109884d05 (as.factor (cols frame_rdd_58_b5fd354222706e76381948c109884d05 [8])) [8] [])), session_id=_sid_8db5}
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: java.lang.NoClassDefFoundError: no/priv/garshol/duke/Comparator
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.rapids.Env.<clinit>(Env.java:278)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.rapids.Session.exec(Session.java:86)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.rapids.Rapids.exec(Rapids.java:93)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.api.RapidsHandler.exec(RapidsHandler.java:41)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at java.lang.reflect.Method.invoke(Method.java:498)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.api.Handler.handle(Handler.java:63)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.api.RequestServer.serve(RequestServer.java:448)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.api.RequestServer.doGeneric(RequestServer.java:297)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.api.RequestServer.doPost(RequestServer.java:223)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at water.JettyHTTPD$LoginHandler.handle(JettyHTTPD.java:183)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.Server.handle(Server.java:370)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at java.lang.Thread.run(Thread.java:748)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: Caused by: java.lang.ClassNotFoundException: no.priv.garshol.duke.Comparator
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
06-19 22:46:25.081 127.0.0.1:54321 2158 #2868-244 ERRR: ... 37 more
1 ответ
Эта ошибка была исправлена в последних версиях Sparkling Water для Spark 2.0, Spark 2.1 и Spark 2.2