Hive学习笔记——metadata

可以在hive的jdbc接口中使用getMetaData方法来获取hive表的相关元信息

statement = connection.createStatement(); DatabaseMetaData meta = connection.getMetaData();

参考

https://blog.csdn.net/u010368839/article/details/76358831

hive metadata源码解析可以参考

https://cloud.tencent.com/developer/article/1330250

hive thrift接口可以参考

http://www.laicar.com/book/echapter/5cb0bcfe739207662ac88ed1/links/x_Chapter_16.html/OEBPS/Text/part0024.xhtml

获得表的信息接口，指定tableNamePattern为hive表名

ResultSet tableRet = meta.getTables(null, "%", "ads_nsh_trade", new String[]{"TABLE"}); while (tableRet.next()) { System.out.println("TABLE_CAT:" + tableRet.getString("TABLE_CAT")); System.out.println("TABLE_SCHEM:" + tableRet.getString("TABLE_SCHEM")); System.out.println("TABLE_NAME => " + tableRet.getString("TABLE_NAME")); System.out.println("table_type => " + tableRet.getString("table_type")); System.out.println("remarks => " + tableRet.getString("remarks")); System.out.println("type_cat => " + tableRet.getString("type_cat")); System.out.println("type_schem => " + tableRet.getString("type_schem")); System.out.println("type_name => " + tableRet.getString("type_name")); System.out.println("self_referencing_col_name => " + tableRet.getString("self_referencing_col_name")); System.out.println("ref_generation => " + tableRet.getString("ref_generation")); }

其中的参数可以是

table_cat, table_schem, table_name, table_type, remarks, type_cat, type_schem, type_name, self_referencing_col_name, ref_generation

如果填写不正确将会抛出异常

java.sql.SQLException: Could not find COLUMN_NAME in [table_cat, table_schem, table_name, table_type, remarks, type_cat, type_schem, type_name, self_referencing_col_name, ref_generation] at org.apache.hive.jdbc.HiveBaseResultSet.findColumn(HiveBaseResultSet.java:100) at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:541)

输出的结果

TABLE_CAT: TABLE_SCHEM:tmp TABLE_NAME => ads_nsh_trade table_type => TABLE remarks => ??????????? type_cat => null type_schem => null type_name => null self_referencing_col_name => null ref_generation => null TABLE_CAT: TABLE_SCHEM:default TABLE_NAME => ads_nsh_trade table_type => TABLE remarks => null type_cat => null type_schem => null type_name => null self_referencing_col_name => null ref_generation => null

如果再指定schemaPattern为hive库名

ResultSet tableRet = meta.getTables(null, "default", "ads_nsh_trade", new String[]{"TABLE"}); while (tableRet.next()) { System.out.println("TABLE_CAT:" + tableRet.getString("TABLE_CAT")); System.out.println("TABLE_SCHEM:" + tableRet.getString("TABLE_SCHEM")); System.out.println("TABLE_NAME => " + tableRet.getString("TABLE_NAME")); System.out.println("table_type => " + tableRet.getString("table_type")); System.out.println("remarks => " + tableRet.getString("remarks")); System.out.println("type_cat => " + tableRet.getString("type_cat")); System.out.println("type_schem => " + tableRet.getString("type_schem")); System.out.println("type_name => " + tableRet.getString("type_name")); System.out.println("self_referencing_col_name => " + tableRet.getString("self_referencing_col_name")); System.out.println("ref_generation => " + tableRet.getString("ref_generation")); }

输出结果

TABLE_CAT: TABLE_SCHEM:default TABLE_NAME => ads_nsh_trade table_type => TABLE remarks => null type_cat => null type_schem => null type_name => null self_referencing_col_name => null ref_generation => null

在hive的元数据表中，表的信息主要在TBLS和TABLE_PARAMS这两张表中

参考

https://blog.csdn.net/haozhugogo/article/details/73274832

比如TBLS表

和TABLE_PARAMS表

获得表的字段信息的接口

ResultSet rs1 = meta.getColumns("default", "%", "ads_nsh_trade", "%"); while (rs1.next()) { String tableCat = rs1.getString("table_cat"); String tableSchem = rs1.getString("table_schem"); String tableName = rs1.getString("table_name"); String columnName = rs1.getString("COLUMN_NAME"); String columnType = rs1.getString("TYPE_NAME"); String remarks = rs1.getString("REMARKS"); int datasize = rs1.getInt("COLUMN_SIZE"); int digits = rs1.getInt("DECIMAL_DIGITS"); int nullable = rs1.getInt("NULLABLE"); System.out.println(tableCat + " " + tableSchem + " " + tableName + " " + columnName + " " + columnType + " " + datasize + " " + digits + " " + nullable + " " + remarks); }

其中的参数可以是

table_cat, table_schem, table_name, column_name, data_type, type_name, column_size, buffer_length, decimal_digits, num_prec_radix, nullable, remarks, column_def, sql_data_type, sql_datetime_sub, char_octet_length, ordinal_position, is_nullable, scope_catalog, scope_schema, scope_table, source_data_type, is_auto_increment

输出的结果

null default ads_nsh_trade test_string STRING 2147483647 0 1 string?????? null default ads_nsh_trade test_boolean BOOLEAN 0 0 1 boolean?????? null default ads_nsh_trade test_short SMALLINT 5 0 1 short?????? null default ads_nsh_trade test_double DOUBLE 15 15 1 double?????? null default ads_nsh_trade test_byte TINYINT 3 0 1 byte?????? null default ads_nsh_trade test_list array<string> 0 0 1 list<String>???? null default ads_nsh_trade test_map map<string,int> 0 0 1 map<String,Int>???? null default ads_nsh_trade test_int INT 10 0 1 int?????? null default ads_nsh_trade test_set array<bigint> 0 0 1 set<Long>?????? null default ads_nsh_trade col_name DECIMAL 10 2 1 null null default ads_nsh_trade col_name2 DECIMAL 10 2 1 null null default ads_nsh_trade test_long BIGINT 19 0 1 null null tmp ads_nsh_trade test_boolean BOOLEAN 0 0 1 boolean?????? null tmp ads_nsh_trade test_short SMALLINT 5 0 1 short?????? null tmp ads_nsh_trade test_double DOUBLE 15 15 1 double?????? null tmp ads_nsh_trade test_byte TINYINT 3 0 1 byte?????? null tmp ads_nsh_trade test_list array<string> 0 0 1 list<String>???? null tmp ads_nsh_trade test_map map<string,int> 0 0 1 map<String,Int>???? null tmp ads_nsh_trade test_int INT 10 0 1 int?????? null tmp ads_nsh_trade test_set array<bigint> 0 0 1 set<Long>?????? null tmp ads_nsh_trade test_long BIGINT 19 0 1 null null tmp ads_nsh_trade test_string STRING 2147483647 0 1 null

 

转载于:https://www.cnblogs.com/tonglin0325/p/11451946.html