Партонирование по времени с Гобблином

Я новичок в Gobblin и просматриваю документ Kafka в HDFS. Теперь я хочу изменить данный пример файла конфигурации задания, чтобы добавить параметр разделения по времени.
Вот как я его модифицирую. Но это не писать ничего.

job.name=GobblinKafkaQuickStart
job.group=GobblinKafka
job.description=Gobblin quick start job for Kafka
job.lock.enabled=false
fs.uri=file:///

kafka.brokers=localhost:9092

source.class=org.apache.gobblin.source.extractor.extract.kafka.KafkaSimpleSource
extract.namespace=org.apache.gobblin.extract.kafka

writer.builder.class=org.apache.gobblin.writer.SimpleDataWriterBuilder
writer.partitioner.class=gobblin.writer.partitioner.SimpleDataWriter
writer.partition.granularity=day
writer.partition.pattern=YYYY-MM-dd
writer.partition.timezone=UTC
writer.file.path.type=tablename
writer.destination.type=HDFS
writer.output.format=txt

data.publisher.type=org.apache.gobblin.publisher.BaseDataPublisher
data.publisher.replace.final.dir=false
data.publisher.final.dir=/destination/dir

mr.job.max.mappers=1

metrics.reporting.file.enabled=true
metrics.log.dir=${gobblin.cluster.work.dir}/metrics
metrics.reporting.file.suffix=txt

bootstrap.with.offset=earliest

Исключение:

2020-05-15 14:36:59 IST ERROR [JobScheduler-0] org.apache.gobblin.scheduler.JobScheduler$NonScheduledJobRunner  629 - Failed to run job GobblinKafkaQuickStart
org.apache.gobblin.runtime.JobException: Failed to run job GobblinKafkaQuickStart
    at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:437)
    at org.apache.gobblin.scheduler.JobScheduler$NonScheduledJobRunner.run(JobScheduler.java:627)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.gobblin.runtime.JobException: Failed to launch and run job GobblinKafkaQuickStart
    at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:489)
    at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:435)
    ... 4 more
Caused by: org.apache.gobblin.runtime.JobException: Job job_GobblinKafkaQuickStart_1589533613945 failed
    at org.apache.gobblin.runtime.AbstractJobLauncher.launchJob(AbstractJobLauncher.java:521)
    at org.apache.gobblin.scheduler.JobScheduler.runJob(JobScheduler.java:479)
    ... 5 more

Что мне не хватает?

0 ответов

Другие вопросы по тегам