PostgreSQL教程FG136-PG数据库元数据查询:表/列/索引信息获取
本文档风哥主要介绍PostgreSQL数据库元数据查询,包括表、列、索引等信息的获取方法,风哥教程参考PostgreSQL官方文档,适合DBA和开发人员在学习和测试中使用。
Part01-基础概念与理论知识
1.1 元数据概述
元数据是描述数据的数据,包括数据库结构、对象定义、约束关系等信息。PostgreSQL通过系统目录和信息模式提供元数据访问接口。更多视频教程www.fgedu.net.cn
- 了解数据库结构
- 生成数据库文档
- 进行数据库审计
- 开发数据库工具
- 数据库迁移和升级
1.2 元数据类型
PostgreSQL中的元数据类型:
from oracle:www.itpux.com
— 1. 结构元数据
— 表、列、索引、约束等结构信息
— 2. 关系元数据
— 表之间的关系、外键关系
— 3. 权限元数据
— 用户权限、角色关系
— 4. 统计元数据
— 表大小、行数、统计信息
— 5. 配置元数据
— 数据库参数、配置信息
1.3 查询方法
查询元数据的主要方法:
- 系统目录:直接查询pg_catalog中的系统表
- 信息模式:查询information_schema中的标准视图
- 系统函数:使用PostgreSQL提供的系统函数
- psql命令:使用psql的元命令
Part02-生产环境规划与建议
2.1 表元数据查询
2.1.1 基本表信息查询
— 1. 查询所有表
SELECT
schemaname,
tablename,
tableowner
FROM pg_tables
WHERE schemaname = ‘public’
ORDER BY tablename;
— 2. 查询表详细信息
SELECT
c.relname as table_name,
c.relkind as table_type,
c.reltuples::bigint as row_count,
pg_size_pretty(pg_total_relation_size(c.oid)) as total_size,
pg_size_pretty(pg_relation_size(c.oid)) as table_size,
pg_size_pretty(pg_indexes_size(c.oid)) as index_size,
obj_description(c.oid, ‘pg_class’) as description
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind = ‘r’
AND n.nspname = ‘public’
ORDER BY pg_total_relation_size(c.oid) DESC;
— 3. 查询表的创建时间
SELECT
relname,
pg_catalog.pg_get_fgedubyid(relowner) as owner,
relcreation::timestamp as creation_time
FROM pg_class
WHERE relkind = ‘r’
AND relnamespace = ‘public’::regnamespace;
— 4. 查询表的主键
SELECT
tc.table_name,
kcu.column_name,
tc.constraint_name
FROM information_schema.table_constraints tc
JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
WHERE tc.constraint_type = ‘PRIMARY KEY’
AND tc.table_schema = ‘public’;
2.2 列元数据查询
2.2.1 列信息查询
— 1. 查询表的所有列
SELECT
column_name,
data_type,
character_maximum_length,
numeric_precision,
numeric_scale,
is_nullable,
column_default,
ordinal_position
FROM information_schema.columns
WHERE table_schema = ‘public’
AND table_name = ‘fgedu_employees’
ORDER BY ordinal_position;
— 2. 查询列的详细信息(系统目录方式)
SELECT
a.attname as column_name,
pg_catalog.format_type(a.atttypid, a.atttypmod) as data_type,
CASE WHEN a.attnotnull THEN ‘NOT NULL’ ELSE ‘NULL’ END as nullable,
pg_get_expr(d.adbin, d.adrelid) as default_value,
col_description(a.attrelid, a.attnum) as description
FROM pg_attribute a
LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = ‘fgedu_employees’::regclass
AND a.attnum > 0
AND NOT a.attisdropped
ORDER BY a.attnum;
— 3. 查询所有表的可空列
SELECT
table_name,
column_name,
data_type
FROM information_schema.columns
WHERE is_nullable = ‘YES’
AND table_schema = ‘public’
ORDER BY table_name, ordinal_position;
— 4. 查询有默认值的列
SELECT
table_name,
column_name,
data_type,
column_default
FROM information_schema.columns
WHERE column_default IS NOT NULL
AND table_schema = ‘public’
ORDER BY table_name, ordinal_position;
2.3 索引元数据查询
2.3.1 索引信息查询
— 1. 查询所有索引
SELECT
schemaname,
tablename,
indexname,
indexdef
FROM pg_indexes
WHERE schemaname = ‘public’
ORDER BY tablename, indexname;
— 2. 查询表的索引
SELECT
indexname,
indexdef,
pg_size_pretty(pg_relation_size(indexrelid)) as index_size
FROM pg_indexes pi
JOIN pg_stat_fgedu_indexes psi ON pi.indexname = psi.indexrelname
WHERE pi.schemaname = ‘public’
AND pi.tablename = ‘fgedu_employees’;
— 3. 查询索引使用情况
SELECT
schemaname,
relname as table_name,
indexrelname as index_name,
idx_scan as index_scans,
idx_tup_read as tuples_read,
idx_tup_fetch as tuples_fetched
FROM pg_stat_fgedu_indexes
WHERE schemaname = ‘public’
ORDER BY idx_scan DESC;
— 4. 查询重复索引
SELECT
tablename,
indexdef,
COUNT(*) as duplicate_count
FROM pg_indexes
WHERE schemaname = ‘public’
GROUP BY tablename, indexdef
HAVING COUNT(*) > 1;
— 5. 查询未使用的索引
SELECT
schemaname,
relname as table_name,
indexrelname as index_name
FROM pg_stat_fgedu_indexes
WHERE idx_scan = 0
AND schemaname = ‘public’
ORDER BY relname;
Part03-生产环境项目实施方案
3.1 高级查询
3.1.1 复杂元数据查询
— 1. 生成表结构文档
SELECT
c.table_name,
c.column_name,
c.data_type ||
CASE
WHEN c.character_maximum_length IS NOT NULL
THEN ‘(‘ || c.character_maximum_length || ‘)’
WHEN c.numeric_precision IS NOT NULL
THEN ‘(‘ || c.numeric_precision || ‘,’ || COALESCE(c.numeric_scale, 0) || ‘)’
ELSE ”
END as full_data_type,
CASE WHEN c.is_nullable = ‘NO’ THEN ‘NOT NULL’ ELSE ” END as nullable,
COALESCE(c.column_default, ”) as default_value,
COALESCE(
(SELECT ‘PK’ FROM information_schema.table_constraints tc
JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
WHERE tc.constraint_type = ‘PRIMARY KEY’
AND kcu.table_name = c.table_name
AND kcu.column_name = c.column_name
LIMIT 1),
”
) as is_primary_key,
COALESCE(
(SELECT ‘FK’ FROM information_schema.table_constraints tc
JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
WHERE tc.constraint_type = ‘FOREIGN KEY’
AND kcu.table_name = c.table_name
AND kcu.column_name = c.column_name
LIMIT 1),
”
) as is_foreign_key
FROM information_schema.columns c
WHERE c.table_schema = ‘public’
ORDER BY c.table_name, c.ordinal_position;
— 2. 查询外键关系图
SELECT
tc.table_name as from_table,
kcu.column_name as from_column,
ccu.table_name as to_table,
ccu.column_name as to_column,
tc.constraint_name
FROM information_schema.table_constraints tc
JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
JOIN information_schema.constraint_column_usage ccu
ON ccu.constraint_name = tc.constraint_name
WHERE tc.constraint_type = ‘FOREIGN KEY’
AND tc.table_schema = ‘public’
ORDER BY tc.table_name;
— 3. 查询数据库对象统计
SELECT
‘Tables’ as object_type,
COUNT(*) as object_count
FROM information_schema.tables
WHERE table_schema = ‘public’
AND table_type = ‘BASE TABLE’
UNION ALL
SELECT
‘Views’,
COUNT(*)
FROM information_schema.tables
WHERE table_schema = ‘public’
AND table_type = ‘VIEW’
UNION ALL
SELECT
‘Indexes’,
COUNT(*)
FROM pg_indexes
WHERE schemaname = ‘public’
UNION ALL
SELECT
‘Functions’,
COUNT(*)
FROM pg_proc p
JOIN pg_namespace n ON p.pronamespace = n.oid
WHERE n.nspname = ‘public’;
3.2 动态查询
3.2.1 动态元数据查询
— 1. 创建动态查询函数
CREATE OR REPLACE FUNCTION get_table_columns(p_table_name text)
RETURNS TABLE (
column_name text,
data_type text,
is_nullable text
) AS $$
BEGIN
RETURN QUERY
SELECT
c.column_name::text,
c.data_type::text,
c.is_nullable::text
FROM information_schema.columns c
WHERE c.table_name = p_table_name
AND c.table_schema = ‘public’
ORDER BY c.ordinal_position;
END;
$$ LANGUAGE plpgsql;
— 使用函数
SELECT * FROM get_table_columns(‘fgedu_employees’);
— 2. 创建通用元数据查询函数
CREATE OR REPLACE FUNCTION get_object_metadata(p_object_type text)
RETURNS TABLE (
object_name text,
object_size text,
object_owner text
) AS $$
BEGIN
IF p_object_type = ‘table’ THEN
RETURN QUERY
SELECT
c.relname::text,
pg_size_pretty(pg_total_relation_size(c.oid))::text,
pg_get_fgedubyid(c.relowner)::text
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind = ‘r’ AND n.nspname = ‘public’;
ELSIF p_object_type = ‘index’ THEN
RETURN QUERY
SELECT
indexname::text,
pg_size_pretty(pg_relation_size(indexrelid))::text,
tableowner::text
FROM pg_indexes pi
JOIN pg_stat_fgedu_indexes psi ON pi.indexname = psi.indexrelname
WHERE pi.schemaname = ‘public’;
END IF;
END;
$$ LANGUAGE plpgsql;
— 使用函数
SELECT * FROM get_object_metadata(‘table’);
— 3. 动态生成SQL
CREATE OR REPLACE FUNCTION generate_select_sql(p_table_name text)
RETURNS text AS $$
DECLARE
v_sql text;
v_columns text;
BEGIN
SELECT STRING_AGG(column_name, ‘, ‘)
INTO v_columns
FROM information_schema.columns
WHERE table_name = p_table_name
AND table_schema = ‘public’;
v_sql := ‘SELECT ‘ || v_columns || ‘ FROM ‘ || p_table_name || ‘;’;
RETURN v_sql;
END;
$$ LANGUAGE plpgsql;
— 使用函数
SELECT generate_select_sql(‘fgedu_employees’);
3.3 查询优化
3.3.1 优化元数据查询
— 1. 使用缓存
CREATE TABLE fgedu_metadata_cache (
cache_key text PRIMARY KEY,
cache_value jsonb,
cached_at timestamp DEFAULT CURRENT_TIMESTAMP
);
— 2. 创建缓存刷新函数
CREATE OR REPLACE FUNCTION refresh_table_metadata_cache()
RETURNS void AS $$
BEGIN
INSERT INTO fgedu_metadata_cache (cache_key, cache_value)
SELECT ‘tables’, jsonb_agg(jsonb_build_object(
‘table_name’, table_name,
‘table_type’, table_type,
‘column_count’, (SELECT COUNT(*) FROM information_schema.columns c
WHERE c.table_name = t.table_name)
))
FROM information_schema.tables t
WHERE table_schema = ‘public’
ON CONFLICT (cache_key) DO UPDATE SET
cache_value = EXCLUDED.cache_value,
cached_at = CURRENT_TIMESTAMP;
END;
$$ LANGUAGE plpgsql;
— 3. 使用物化视图
CREATE MATERIALIZED VIEW fgedu_table_metadata AS
SELECT
t.table_name,
t.table_type,
COUNT(c.column_name) as column_count,
COUNT(CASE WHEN c.is_nullable = ‘NO’ THEN 1 END) as not_null_count
FROM information_schema.tables t
LEFT JOIN information_schema.columns c
ON t.table_name = c.table_name AND t.table_schema = c.table_schema
WHERE t.table_schema = ‘public’
GROUP BY t.table_name, t.table_type;
— 刷新物化视图
REFRESH MATERIALIZED VIEW fgedu_table_metadata;
— 4. 创建索引
CREATE INDEX idx_fgedu_table_metadata_name ON fgedu_table_metadata(table_name);
Part04-生产案例与实战讲解
4.1 常见用例
4.1.1 数据库文档生成
— 1. 生成表结构文档
CREATE OR REPLACE FUNCTION generate_table_documentation()
RETURNS TABLE (
table_name text,
column_name text,
data_type text,
nullable text,
default_value text,
description text
) AS $$
BEGIN
RETURN QUERY
SELECT
c.table_name::text,
c.column_name::text,
c.data_type::text,
c.is_nullable::text,
COALESCE(c.column_default, ”)::text,
col_description((c.table_schema || ‘.’ || c.table_name)::regclass::oid, c.ordinal_position)::text
FROM information_schema.columns c
WHERE c.table_schema = ‘public’
ORDER BY c.table_name, c.ordinal_position;
END;
$$ LANGUAGE plpgsql;
— 2. 生成ER图数据
CREATE OR REPLACE FUNCTION generate_er_diagram_data()
RETURNS TABLE (
from_table text,
from_column text,
to_table text,
to_column text,
relationship_type text
) AS $$
BEGIN
RETURN QUERY
SELECT
tc.table_name::text,
kcu.column_name::text,
ccu.table_name::text,
ccu.column_name::text,
‘FOREIGN KEY’::text
FROM information_schema.table_constraints tc
JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
JOIN information_schema.constraint_column_usage ccu
ON ccu.constraint_name = tc.constraint_name
WHERE tc.constraint_type = ‘FOREIGN KEY’
AND tc.table_schema = ‘public’;
END;
$$ LANGUAGE plpgsql;
— 3. 生成数据字典
CREATE OR REPLACE FUNCTION generate_data_dictionary()
RETURNS TABLE (
table_name text,
table_description text,
column_count bigint,
row_count bigint,
table_size text
) AS $$
BEGIN
RETURN QUERY
SELECT
t.table_name::text,
obj_description((t.table_schema || ‘.’ || t.table_name)::regclass::oid, ‘pg_class’)::text,
(SELECT COUNT(*) FROM information_schema.columns c
WHERE c.table_name = t.table_name)::bigint,
(SELECT reltuples::bigint FROM pg_class WHERE relname = t.table_name),
pg_size_pretty(pg_total_relation_size((t.table_schema || ‘.’ || t.table_name)::regclass))::text
FROM information_schema.tables t
WHERE t.table_schema = ‘public’
AND t.table_type = ‘BASE TABLE’
ORDER BY t.table_name;
END;
$$ LANGUAGE plpgsql;
4.2 自动化脚本
4.2.1 元数据收集脚本
# 元数据收集脚本
# from:www.itpux.com.qq113257174.wx:itpux-com
# from:www.itpux.com.qq113257174.wx:itpux-com
# from:www.itpux.com.qq113257174.wx:itpux-com
# from:www.itpux.com.qq113257174.wx:itpux-com
# web: `http://www.fgedu.net.cn`
# web: `http://www.fgedu.net.cn`
# web: `http://www.fgedu.net.cn`
# web: http://www.fgedu.net.cn
DB_NAME=”fgedudb”
DB_USER=”pgsql”
OUTPUT_DIR=”/tmp/db_metadata/$(date +%Y%m%d)”
mkdir -p $OUTPUT_DIR
echo “开始收集元数据…”
# 1. 收集表信息
echo “收集表信息…”
psql -U $DB_USER -d $DB_NAME -c ”
COPY (
SELECT
schemaname,
tablename,
tableowner,
pg_size_pretty(pg_total_relation_size(schemaname || ‘.’ || tablename)) as size
FROM pg_tables
WHERE schemaname = ‘public’
ORDER BY pg_total_relation_size(schemaname || ‘.’ || tablename) DESC
) TO ‘$OUTPUT_DIR/tables.csv’ WITH CSV HEADER;
”
# 2. 收集列信息
echo “收集列信息…”
psql -U $DB_USER -d $DB_NAME -c ”
COPY (
SELECT
table_name,
column_name,
data_type,
is_nullable,
column_default
FROM information_schema.columns
WHERE table_schema = ‘public’
ORDER BY table_name, ordinal_position
) TO ‘$OUTPUT_DIR/columns.csv’ WITH CSV HEADER;
”
# 3. 收集索引信息
echo “收集索引信息…”
psql -U $DB_USER -d $DB_NAME -c ”
COPY (
SELECT
schemaname,
tablename,
indexname,
pg_size_pretty(pg_relation_size(indexrelid)) as size
FROM pg_indexes pi
JOIN pg_stat_fgedu_indexes psi ON pi.indexname = psi.indexrelname
WHERE pi.schemaname = ‘public’
ORDER BY pg_relation_size(indexrelid) DESC
) TO ‘$OUTPUT_DIR/indexes.csv’ WITH CSV HEADER;
”
# 4. 生成报告
echo “生成报告…”
cat > $OUTPUT_DIR/report.txt << EOF
数据库元数据报告
生成时间: $(date)
数据库: $DB_NAME
表数量: $(psql -U $DB_USER -d $DB_NAME -t -c "SELECT COUNT(*) FROM pg_tables WHERE schemaname = 'public'")
索引数量: $(psql -U $DB_USER -d $DB_NAME -t -c "SELECT COUNT(*) FROM pg_indexes WHERE schemaname = 'public'")
总大小: $(psql -U $DB_USER -d $DB_NAME -t -c "SELECT pg_size_pretty(SUM(pg_total_relation_size(schemaname || '.' || tablename))) FROM pg_tables WHERE schemaname = 'public'")
EOF
echo "元数据收集完成,保存在: $OUTPUT_DIR"
4.3 实战案例
4.3.1 数据库健康检查
— 1. 检查无主键的表
SELECT table_name
FROM information_schema.tables
WHERE table_schema = ‘public’
AND table_type = ‘BASE TABLE’
AND table_name NOT IN (
SELECT table_name FROM information_schema.table_constraints
WHERE constraint_type = ‘PRIMARY KEY’
);
— 2. 检查无索引的表
SELECT tablename
FROM pg_tables
WHERE schemaname = ‘public’
AND tablename NOT IN (
SELECT tablename FROM pg_indexes WHERE schemaname = ‘public’
);
— 3. 检查大表(超过100万行)
SELECT
relname as table_name,
reltuples::bigint as row_count,
pg_size_pretty(pg_total_relation_size(oid)) as size
FROM pg_class
WHERE relkind = ‘r’
AND relnamespace = ‘public’::regnamespace
AND reltuples > 1000000
ORDER BY reltuples DESC;
— 4. 检查未使用的索引
SELECT
schemaname,
relname as table_name,
indexrelname as index_name
FROM pg_stat_fgedu_indexes
WHERE idx_scan = 0
AND schemaname = ‘public’
ORDER BY relname;
— 5. 检查重复索引
SELECT
tablename,
indexdef,
COUNT(*) as count
FROM pg_indexes
WHERE schemaname = ‘public’
GROUP BY tablename, indexdef
HAVING COUNT(*) > 1;
Part05-风哥经验总结与分享
5.1 最佳实践
元数据查询最佳实践:
- 使用缓存:对于不经常变化的元数据,使用缓存提高性能
- 定期收集:建立定期元数据收集机制
- 自动化:使用脚本自动化元数据查询
- 文档化:记录元数据查询结果,生成文档
- 监控告警:建立元数据变化监控和告警
5.2 性能技巧
— 1. 限制返回行数
SELECT * FROM information_schema.columns
WHERE table_schema = ‘public’
LIMIT 100;
— 2. 使用索引列过滤
SELECT * FROM pg_class
WHERE relnamespace = ‘public’::regnamespace;
— 3. 避免SELECT *
SELECT column_name, data_type FROM information_schema.columns
WHERE table_schema = ‘public’;
— 4. 使用物化视图缓存结果
CREATE MATERIALIZED VIEW mv_table_info AS
SELECT * FROM information_schema.tables
WHERE table_schema = ‘public’;
— 5. 定期刷新缓存
REFRESH MATERIALIZED VIEW CONCURRENTLY mv_table_info;
5.3 故障排除
元数据查询常见问题:
- 权限不足:确保用户有权限访问系统目录
- 查询缓慢:添加过滤条件,使用缓存
- 信息不准确:ANALYZE更新统计信息
- 版本差异:注意不同版本的系统目录结构差异
本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html
