This post does not discuss mutexes or latches. I wrote it under the impression of recent escalation due to database corruption.
This was Oracle 10.2.0.4 database on HP-UX. Database files underwent recurrent corruption. Most commonly blocks were corrupted in system and undo tablespaces. This resulted in corrupted dictionary tables and indexes. Neither db_block_checking/db_block_checksum nor installation of latest PSU and OS patches helped. SCNs showed that these dictionary blocks were not changed for years. It looks likes some program other than DBWR overwrites the blocks. Corruptions caused downtimes to restore and recover datafiles.
The corruption always started with the same byte pattern: “00 0B 00 00 0C 00 00 00 01 00 01”. This not resembled any known for me Oracle database structure. Many thanks to customer DBA, who found the same bytes in discussion of network sniffer output. It was identified as NSPTMK packet. Being definitely related to Oracle SQL*Net, what it is doing in the middle of data block?
MOS search found interesting Bug 8943287 ORA-1578 corrupt block with SQL*Net AUTH strings [ID 976852.1]. The note states that SQL*Net may overwrite database files with sqlnet packets when sqlnet.inbound_connect_timeout is greater than 0 in the server sqlnet.ora file (the default value is 60).
This mean that under some (rare) circumstances, the shadow Oracle process may mix up its file descriptors and write() the next SQL*Net package … into file instead of socket. Such sniper shot will corrupt one of previously opened datafiles starting from the position of last read. This may be the controlfile also.
If you are running pre-220.127.116.11 Oracle database on any *nix except Solaris, look on your server sqlnet.ora file. If it is absent or not contain sqlnet.inbound_connect_timeout=0 then your database may be vulnerable to this problem. It is worth to apply the patch 8943287 or set sqlnet.inbound_connect_timeout=0 as workaround. Such workaround may require increase of processes parameter.
This problem has been named “most up-to-date high impact and urgent issue” in note
“Information Center: Oracle Net Services” [ID 1381244.2].