Goldengate一个令人郁闷的小细节

晚上有兴趣测试了下Goldengate的initial load功能,通过initial load+change sync可以很容易做到数据迁移data migration;不过发现一个令人很郁闷的细节,就是replicat的params中定义map参数时TARGET到前面一个逗号间要留一个空格space:

MAP table spec, TARGET table spec     
/*TARGET到前面一个逗号间要留一个空格space*/
[, DEF ]
[, TARGETDEF ]
[, COLMAP ()]
[, EVENTACTIONS ()]
[, EXCEPTIONSONLY]
[, EXITPARAM “”]
[, FILTER ()]
[, HANDLECOLLISIONS | NOHANDLECOLLISIONS]
[, INSERTALLRECORDS]
[, INSERTAPPEND | NOINSERTAPPEND]
[, KEYCOLS ()]
[, REPERROR ( , )]
[, SQLEXEC ()]
[, TRIMSPACES | NOTRIMSPACES]
[, WHERE ()]
;

如果map参数中target和前面的逗号间没有空格会出现invaild option for map错误:

GGSCI (rh2.oracle.com) 32> view params init1
extract init1
userid ggate,password ggate
RMTHOST rh3.oracle.com,MGRPORT 7809
RMTTASK replicat,GROUP  init3
table sh.products;

GGSCI (rh3.oracle.com) 40> view params init3

replicat init3
userid ggate,password ggate
ASSUMETARGETDEFS
map sh.products,target sh.products;

GGSCI (rh2.oracle.com) 33> start extract init1

Sending START request to MANAGER ...
EXTRACT INIT1 starting

GGSCI (rh3.oracle.com) 41> view report init3
..............
MAP resolved (entry SH.PRODUCTS):
  map SH.PRODUCTS, sh.products;

Source Context :
  SourceModule            : [er.main]
  SourceID                : [/home/ecloud/workspace/Build_FBO_OpenSys_r11.1.1.0.0_078_[34093]/perforce/src/app/er/rep.c]
  SourceFunction          : [get_map_entry]
  SourceLine              : [8573]
  ThreadBacktrace         : [11] elements
                          : [./replicat(CMessageContext::AddThreadContext()+0x26) [0x5d9516]]
                          : [./replicat(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x7b2) [0x5cffb2]]
                          : [./replicat(_MSG_ERR_STARTUP_PARAMERROR_INVALID_OPTION_VALUE
(CSourceContext*, char const*, char const*, CMessageFactory::MessageDisposition)
+0x9b) [0x59086b]]
                          : [./replicat(get_map_entry(char*, int, __wc*, int)+0x284d) [0x4ee66d]]
                          : [./replicat [0x5357d5]]
                          : [./replicat(WILDCARD_check_table(char const*, char const*, int, 
unsigned int*, int, unsigned int, DBString<777>*, int)+0x15e) [0x536fce]]
                          : [./replicat(REP_find_source_file_wc(char const*, unsigned int, DBString<777>*, int)+0x64f) [0x8a1caf]]
                          : [./replicat [0x8a9b04]]
                          : [./replicat(main+0x227a) [0x4f50da]]
                          : [/lib64/libc.so.6(__libc_start_main+0xf4) [0x340d01d994]]
                          : [./replicat(__gxx_personality_v0+0x1e2) [0x4d86ba]]

2010-12-01 08:58:30  ERROR   OGG-00212  Invalid option for MAP: sh.products.

/*我们来加上空格*/

GGSCI (rh3.oracle.com) 43> view params init3
replicat init3
userid ggate,password ggate
ASSUMETARGETDEFS
map sh.products, target sh.products;

GGSCI (rh3.oracle.com) 43> view report init3
MAP resolved (entry SH.PRODUCTS):
  map SH.PRODUCTS, target sh.products;

2010-12-01 09:00:30  WARNING OGG-00869  No unique key is defined for table PRODUCTS. 
All viable columns will be used to represent the key, but may not guarantee uniqueness.  
KEYCOLS may be used to define the key.
Using following columns in default map by name:
  PROD_ID, PROD_NAME, PROD_DESC, PROD_SUBCATEGORY, PROD_SUBCATEGORY_ID, 
  PROD_SUBCATEGORY_DESC, PROD_CATEGORY, PROD_CATEGORY_ID, 
  PROD_CATEGORY_DESC, PROD_WEIGHT_CLASS, PROD_UNIT_OF_MEASURE, 
  PROD_PACK_SIZE, SUPPLIER_ID, PROD_STATUS, PROD_LIST_PRICE, 
  PROD_MIN_PRICE, PROD_TOTAL, PROD_TOTAL_ID, PROD_SRC_ID, 
  PROD_EFF_FROM, PROD_EFF_TO, PROD_VALID

/* 加上空格后初始化成功*/

11g内存管理新特性的internal表现

11g中自动内存管理(Automatic Memory Management ,amm), 令dba在数据库内存配置的相关工作更加简单. AMM现在将SGA与PGA整合到一起管理,而您只需要设置memory_target参数即可限定Oracle将使用到的内存尺寸,Oracle将自动分配这些内存空间.

您一定很困惑Oracle在unix平台上是如何对共享的sga内存空间与私有的pga内存空间进行切换的?这意味着Oracle需要经常释放sga中的部分内存以便允许pga去使用它们.传统的sys V 使用的共享内存shm接口不具备如此的灵活性.我们来看看Oracle是如何做到的?

先来获取我们需要的11g实例共享内存id(shared memory id)

[oracle@rh2 ~]$ sysresv                // 该命令需要设置了正确的LD_LIBRARY_PATH
IPC Resources for ORACLE_SID “T11” :
Shared Memory:
ID              KEY
65537           0x95c84bb8
Semaphores:
ID              KEY
327681          0xdf521034
Oracle Instance alive for sid “T11”

试着找出对应的sys V共享内存段:

[oracle@rh2 ~]$ ipcs -m

—— Shared Memory Segments ——–
key        shmid      owner      perms      bytes      nattch     status
0x95c84bb8 65537      oracle    660        4096       0

对应的存在着共享内存段,但该段很小只有 4096 byte哦,既然Oracle不再把sga放到共享段中,那藏到哪里去了呢?

我们接下来检查Oracle实例进程的内存影射状况.

[oracle@rh2 ~]$  pmap `pgrep -f lgwr`|less
14889:   ora_lgwr_T11
0000000000400000 155016K r-x–  /usr/oracle/product/11g/db_1/bin/oracle
0000000009c62000  12404K rw—  /usr/oracle/product/11g/db_1/bin/oracle
000000000a87f000    732K rwx–    [ anon ]
0000000060000000      4K r–s-  /dev/shm/ora_T11_65537_0
0000000060001000  16380K rw-s-  /dev/shm/ora_T11_65537_0
0000000061000000  16384K rw-s-  /dev/shm/ora_T11_65537_1
0000000062000000  16384K rw-s-  /dev/shm/ora_T11_65537_2
0000000063000000  16384K rw-s-  /dev/shm/ora_T11_65537_3
0000000064000000  16384K rw-s-  /dev/shm/ora_T11_65537_4
0000000065000000  16384K rw-s-  /dev/shm/ora_T11_65537_5
0000000066000000  16384K rw-s-  /dev/shm/ora_T11_65537_6
0000000067000000  16384K rw-s-  /dev/shm/ora_T11_65537_7
0000000068000000  16384K rw-s-  /dev/shm/ora_T11_65537_8
0000000069000000  16384K rw-s-  /dev/shm/ora_T11_65537_9
000000006a000000  16384K rw-s-  /dev/shm/ora_T11_65537_10
000000006b000000  16384K rw-s-  /dev/shm/ora_T11_65537_11
000000006c000000  16384K rw-s-  /dev/shm/ora_T11_65537_12
000000006d000000  16384K rw-s-  /dev/shm/ora_T11_65537_13
000000006e000000  16384K rw-s-  /dev/shm/ora_T11_65537_14
000000006f000000  16384K rw-s-  /dev/shm/ora_T11_65537_15
0000000070000000  16384K rw-s-  /dev/shm/ora_T11_65537_16
0000000071000000  16384K rw-s-  /dev/shm/ora_T11_65537_17
0000000072000000  16384K rw-s-  /dev/shm/ora_T11_65537_18
0000000073000000  16384K rw-s-  /dev/shm/ora_T11_65537_19
0000000074000000  16384K rw-s-  /dev/shm/ora_T11_65537_20
0000000075000000  16384K rw-s-  /dev/shm/ora_T11_65537_21
0000000076000000  16384K rw-s-  /dev/shm/ora_T11_65537_22
0000000077000000  16384K rw-s-  /dev/shm/ora_T11_65537_23
0000000078000000  16384K rw-s-  /dev/shm/ora_T11_65537_24
0000000079000000  16384K rw-s-  /dev/shm/ora_T11_65537_25
000000007a000000  16384K rw-s-  /dev/shm/ora_T11_65537_26
000000007b000000  16384K rw-s-  /dev/shm/ora_T11_65537_27
000000007c000000  16384K rw-s-  /dev/shm/ora_T11_65537_28
000000007d000000  16384K rw-s-  /dev/shm/ora_T11_65537_29
000000007e000000  16384K rw-s-  /dev/shm/ora_T11_65537_30
000000007f000000  16384K rw-s-  /dev/shm/ora_T11_65537_31
0000000080000000  16384K rw-s-  /dev/shm/ora_T11_65537_32
0000000081000000  16384K rw-s-  /dev/shm/ora_T11_65537_33
0000000082000000  16384K rw-s-  /dev/shm/ora_T11_65537_34
0000000083000000  16384K rw-s-  /dev/shm/ora_T11_65537_35
0000000084000000  16384K rw-s-  /dev/shm/ora_T11_65537_36
0000000085000000  16384K rw-s-  /dev/shm/ora_T11_65537_37
0000000086000000  16384K rw-s-  /dev/shm/ora_T11_65537_38
0000000087000000  16384K rw-s-  /dev/shm/ora_T11_65537_39
0000000088000000  16384K rw-s-  /dev/shm/ora_T11_65537_40
0000000089000000  16384K rw-s-  /dev/shm/ora_T11_65537_41
000000008a000000  16384K rw-s-  /dev/shm/ora_T11_65537_42

............

0000003e79109000      4K rw—  /lib64/tls/librt-2.3.4.so
0000003e7910a000     64K rw—    [ anon ]
0000003e79600000     84K r-x–  /lib64/libnsl-2.3.4.so
0000003e79615000   1020K —–  /lib64/libnsl-2.3.4.so
0000003e79714000      4K r—-  /lib64/libnsl-2.3.4.so
0000003e79715000      4K rw—  /lib64/libnsl-2.3.4.so
0000003e79716000      8K rw—    [ anon ]
0000007fbfff3000     52K rwx–    [ stack ]
ffffffffff600000      4K r-x–    [ anon ]
total          2497724K

pmap工具诠释了进程相关共享内存的情况,可以看到许多个16MB的"文件"对应了Oracle服务进程的空间地址.这是linux上POSIX风格的共享内存管理模式,使用"文件"形式包含共享内存段.

借助于将sga分割成许多块,Oracle可以很容易地把sga部分内存返回给OS,而服务器进程即可以利用到这些内存.(当memory_max_target>1024时,颗粒为16MB,否则为4MB).

接下来我们测试下Oracle是如何释放部分sga内存的.

对比实例启动前后:

启动前:

[oracle@rh2 ~]$ ls -l /dev/shm
总用量 0

启动后:
[oracle@rh2 ~]$ ls -l /dev/shm
总用量 1373704
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_0
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_1
-rw-r—–  1 oracle oinstall        0  9月 27 18:59 ora_T11_327680_10
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_100
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_101
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_102
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_103
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_104
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_105
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_106
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_107
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_108
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_109
-rw-r—–  1 oracle oinstall        0  9月 27 18:59 ora_T11_327680_11
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_110
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_111
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_112
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_113
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_114
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_115
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_116
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_117
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_118
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_119
-rw-r—–  1 oracle oinstall        0  9月 27 18:59 ora_T11_327680_12

可以看到启动后出现的16MB文件形式共享内存中部分大小为0,这些块被选出当发生内存交换时来被’destory’.使用pmap工具仍可以看到该部分影射,而实际上已经被Oracle释放了.

现在我们加大pga,观察其交换情况.

SQL> alter system set pga_aggregate_target=1900M ;

System altered.

[oracle@rh2 ~]$ ls -l /dev/shm
总用量 289984
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_0
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_1
-rw-r—–  1 oracle oinstall        0  9月 27 18:59 ora_T11_327680_10
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_100
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_101
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_102
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_103
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_104
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_105
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_106
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_107
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_108
-rw-r—–  1 oracle oinstall 16777216  9月 27 18:59 ora_T11_327680_109
-rw-r—–  1 oracle oinstall        0  9月 27 18:59 ora_T11_327680_11
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_110
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_111
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_112
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_113
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_114
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_115
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_116
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_117
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_118
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_119
-rw-r—–  1 oracle oinstall        0  9月 27 18:59 ora_T11_327680_12
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_120
-rw-r—–  1 oracle oinstall        0  9月 27 19:09 ora_T11_327680_121

可以看到出现了大量size为0的"文件",期许的交换出现了.

可见在11g中Oracle采用了新的共享内存实现方式,区别于旧的"一块式"共享段,更为灵活了.

沪ICP备14014813号

沪公网安备 31010802001379号