Партонирование по времени с Гобблином
Я новичок в Gobblin и просматриваю документ Kafka в HDFS. Теперь я хочу изменить данный пример файла конфигурации задания, чтобы добавить параметр разделения по времени.
Вот как я его модифицирую. Но это не писать ничего.
job.name=GobblinKafkaQuickStart
job.group=GobblinKafka
job.description=Gobblin quick start job for Kafka
job.lock.enabled=false
fs.uri=file:///
kafka.brokers=localhost:9092
source.class=org.apache.gobblin.source.extractor.extract.kafka.KafkaSimpleSource
extract.namespace=org.apache.gobblin.extract.kafka
writer.builder.class=org.apache.gobblin.writer.SimpleDataWriterBuilder
writer.partitioner.class=gobblin.writer.partitioner.SimpleDataWriter
writer.partition.granularity=day
writer.partition.pattern=YYYY-MM-dd
writer.partition.timezone=UTC
writer.file.path.type=tablename
writer.destination.type=HDFS
writer.output.format=txt
data.publisher.type=org.apache.gobblin.publisher.BaseDataPublisher
data.publisher.replace.final.dir=false
data.publisher.final.dir=/destination/dir
mr.job.max.mappers=1
metrics.reporting.file.enabled=true
metrics.log.dir=${gobblin.cluster.work.dir}/metrics
metrics.reporting.file.suffix=txt
bootstrap.with.offset=earliest
Исключение:
2020-05-15 14:36:59 IST ERROR [JobScheduler-0] org.apache.gobblin.scheduler.JobScheduler$NonScheduledJobRunner 629 - Failed to run job GobblinKafkaQuickStart
org.apache.gobblin.runtime.JobException: Failed to run job GobblinKafkaQuickStart
at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:437)
at org.apache.gobblin.scheduler.JobScheduler$NonScheduledJobRunner.run(JobScheduler.java:627)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.gobblin.runtime.JobException: Failed to launch and run job GobblinKafkaQuickStart
at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:489)
at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:435)
... 4 more
Caused by: org.apache.gobblin.runtime.JobException: Job job_GobblinKafkaQuickStart_1589533613945 failed
at org.apache.gobblin.runtime.AbstractJobLauncher.launchJob(AbstractJobLauncher.java:521)
at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:479)
... 5 more
Что мне не хватает?