0

I use the KAFKA JDBC Source connector to read from the database ClickHouse (driver - clickhouse-jdbc-0.2.4.jar) with incrementing mod. Settings:

    batch.max.rows = 100
    catalog.pattern = null
    connection.attempts = 3
    connection.backoff.ms = 10000
    connection.password = [hidden]
    connection.url = jdbc:clickhouse://<ip>:8123/<schema>
    connection.user = user
    db.timezone = 
    dialect.name = 
    incrementing.column.name = id
    mode = incrementing
    numeric.mapping = null
    numeric.precision.mapping = false
    poll.interval.ms = 5000
    query = 
    query.suffix = 
    quote.sql.identifiers = never
    schema.pattern = null
    table.blacklist = []
    table.poll.interval.ms = 60000
    table.types = [TABLE]
    table.whitelist = [<table_name>]
    tables = [default.<schema>.<table_name>]
    timestamp.column.name = []
    timestamp.delay.interval.ms = 0
    timestamp.initial = null
    topic.prefix = staging-
    validate.non.null = false

Why does the connector additionally substitute the default scheme? and how to avoid it?

Instead of a request

SELECT * FROM <schema>.<table_name> WHERE <schema>.<table_name>.id > ? ORDER BY <schema>.<table_name>.id ASC

I get an error with

SELECT * FROM default.<schema>.<table_name> WHERE default.<schema>.<table_name>.id > ? ORDER BY default.<schema>.<table_name>.id ASC

2
  • Where you create the table in default schema or any specific schema? Commented Jun 21, 2021 at 9:03
  • in a specific scheme. I write it in cinnection.url. jdbc:clickhouse://<ip>:8123/<schema>
    – asanisimov
    Commented Jun 21, 2021 at 9:25

1 Answer 1

0

You can create CH data source object like below (Where schema name is not passed).

final ClickHouseDataSource dataSource = new ClickHouseDataSource(
        "jdbc:clickhouse://"+host+"/"+user+"?option1=one%20two&option2=y"); 

Then in SQL query, you can specify a schema name(schema.table). So it will not add the default schema in your query.

1
  • The fact of the matter is that I do not want to ask the sql query. I want to specify the schema and N tables in the settings, so that JDBC Connector himself builds queries to the desired schema. For now, I have to do as you described, and create a separate connector for each table.
    – asanisimov
    Commented Jun 25, 2021 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.