Network Interface No Longer Operational?

<p><strong>Network Interface No Longer Operational' Message in the Alert.log File while NIC Is Running Fine</strong> </p> <h2>Applies to:</h2> Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 10.2.0.5<br> Oracle Net Services - Version: 10.2.0.1 to 10.2.0.5<br> Sun Solaris SPARC (64-bit)<br> <h2><a name="SYMPTOM" rel="nofollow"></a>Symptoms</h2> -- Problem Statement:<br> The following messages are seen in the alert.log file :<br> <br> Tue Dec 16 10:01:55 2008<br> ospid 6526: network interface with IP address 192.168.83.64 no longer operational<br> requested interface 192.168.83.64 ioctl get mtu. Check output from ifconfig command<br> <br> These messages repeat periodically at random intervals. <br> At the time when these messages are printed, it is verified that there is no problem with the network interface in the message.<br> <br> <br> <h2><a name="CAUSE" rel="nofollow"></a>Cause</h2> This is a known Sun Solaris bug : Bug ID 6546482 - Synopsis ioctl call fails with EINTR<br> Reference :<br> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6546482<br> http://sunsolve.sun.com/search/document.do?assetkey=1-1-6546482-1<br> <br> RDS ioctl call is failing with EINTR.<br> <br> <h2><a name="FIX" rel="nofollow"></a>Solution</h2> -- To implement the solution, please execute the following steps::<br> Contact the OS Vendor for providing the fix for Sun Bug ID 6546482<br> <p> </p>

<p><strong>Oracle Cluster process logs "network interface with IP address a.b.c.d no longer operational" message</strong> </p> <p> <strong>Symptoms</strong><br> Oracle(R) RAC Cluster process issues messages similar to the following :<br> <br> Mon Jun 8 10:04:22 2009 ospid 9966: network interface with IP address 172.23.45.67 no longer operational<br> Mon Jun 8 10:05:22 2009 ospid 9966: network interface with IP address 172.23.45.67 is now running<br> 172.23.45.67 being the address of an interconnect interface.<br> <br> Frequency is about twice a month.<br> No further impact observed, all services run normally.<br> <br> <strong>Resolution</strong><br> Root cause is the signal handling within the ora_lmd process.<br> This is being addressed in Oracle Bug ID: 8660487 at this time.<br> <br> <strong>Relief/Workaround</strong><br> There is no further impact to the application (other than above messages being printed).<br> <br> <strong>Additional Information</strong><br> The root cause could be found after writing a little test program issuing<br> the respective ioctl() system calls and by using a debug binary ora_lmd<br> collecting the corresponding errno when the ioctl() system call returned<br> an error.<br> <br> <strong>Product</strong><br> Oracle Real Application Clusters<br> <br> <br> Oracle in Infiniband. Here, as a fix for Oracle(R) RDS, signals were masked<br> while being in the rds_dl_info() function. On a multi-tasking system this<br> lowers probability of this situation but cannot completely exclude it.<br> <br> Furthermore it's special for the Infiniband RDS interface for Oracle.<br> <br> As a more general fix, Oracle will look at the signal handling in the<br> ora_lmd process.<br> <br> Progress and status of this is documented in Oracle Bug ID: 8660487.<br> <br> References :<br> Sun SR <a href="https://webescprod.central.sun.com:8080/servlet/com.sun.webesc.control.IbisDisplay?buttonAction=12&SrNumber=71139694&buttonAction=12" rel="nofollow">71139694</a><br> Sun CR <a href="/CSP/main/article?cmd=show&type=BUG&id=6546482" rel="nofollow">SUNBUG: 6546482</a> ioctl call fails with EINTR<br> Oracle SR 7372738.992<br> Oracle Bug ID: 8660487 </p>

<p><strong>Bug 6546482: ioctl call fails with EINTR</strong> </p> <h3>Fix Request Attributes:</h3> <table> <tbody> <tr> <td>Fix Request #</td> <td>6546482/solaris_nevada</td> </tr> <tr> <td>Target</td> <td>solaris_nevada</td> </tr> <tr> <td>Customer Status</td> <td>10-Fix Delivered</td> </tr> <tr> <td>Severity</td> <td>3-Medium</td> </tr> <tr> <td>Duplicate Of</td> <td></td> </tr> <tr> <td>Committed Version</td> <td>snv_65</td> </tr> <tr> <td>Fixed Version</td> <td>snv_65</td> </tr> <tr> <td>Integrated Version</td> <td>snv_65</td> </tr> <tr> <td>Verified Version</td> <td></td> </tr> <tr> <td>Last Updated</td> <td>2010-04-03 00:00:00 GMT+00:00</td> </tr> </tbody> </table> <br> <h3>Description:</h3> Oracle processes output the following messages when running with RDS:<br> <br> Fri Mar 30 23:01:16 2007<br> ospid 184674: network interface with IP address 172.16.2.1 no longer operational<br> Fri Mar 30 23:02:16 2007<br> ospid 184674: network interface with IP address 172.16.2.1 is now running<br> <br> These messages repeat periodically at random intervals. At the time when these messages are printed, it is verified that there is no problem with the network interface in the message.<br> <br> It has been identified that the RDS ioctl call was failing with EINTR. <table> <tbody> <tr> <td>Date Modified</td> <td>2007-04-17 00:00:00 GMT+00:00</td> </tr> </tbody> </table> <br> <h3>Suggested Fix:</h3> $ sccs diffs rds_ioctl.c<br> <br> ------- rds_ioctl.c -------<br> 93a94<br> > k_sigset_t smask;<br> 105a107<br> > sigintr(&smask, 0);<br> 106a109<br> > sigunintr(&smask);<br> 109a113<br> > sigunintr(&smask);<br> 111a116<br> > sigunintr(&smask);<br> 134c139,144<br> < static boolean_t<br> ---<br> > /*<br> > * Return 0 if the interface is IB.<br> > * Return error (>0) if any error is encountered during process.<br> > * Return -1 if the interface is not IB and no error.<br> > */<br> > static int<br> 159c169<br> < return (B_FALSE);<br> ---<br> > return (-1);<br> 166c176<br> < return (B_TRUE);<br> ---<br> > return (0);<br> 173c183<br> < return (B_FALSE);<br> ---<br> > return (ret);<br> 177d186<br> < <br> 178a188,190<br> > if (ret != 0) {<br> > return (ret);<br> > }<br> 180,182c192,194<br> < if (ret != 0 || (info.dl_mac_type != DL_IB &&<br> < !rds_transport_ops->rds_transport_if_lookup_by_name(name))) {<br> < return (B_FALSE);<br> ---<br> > if (info.dl_mac_type != DL_IB &&<br> > !rds_transport_ops->rds_transport_if_lookup_by_name(name)) {<br> > return (-1);<br> 185c197<br> < return (B_TRUE);<br> ---<br> > return (0);<br> 229c241,242<br> < if (rds_is_ib_interface(ifr->ifr_name)) {<br> ---<br> > err = rds_is_ib_interface(ifr->ifr_name);<br> > if (err == 0) {<br> 230a244,248<br> > } else if (err > 0) {<br> > num_ifs = 0;<br> > break;<br> > } else {<br> > err = 0;<br> 280c298,299<br> < if (rds_is_ib_interface(ifr->ifr_name)) {<br> ---<br> > err = rds_is_ib_interface(ifr->ifr_name);<br> > if (err == 0) {<br> 284a304,307<br> > } else if (err > 0) {<br> > break;<br> > } else {<br> > err = 0;<br> 434c457<br> < rds_is_ib_interface(ifr->ifr_name)) {<br> ---<br> > (rds_is_ib_interface(ifr->ifr_name) == 0)) {<br> $ <table> <tbody> <tr> <td>Date Modified</td> <td>2007-04-16 00:00:00 GMT+00:00</td> </tr> </tbody> </table> <br> <h3>Fix Request Attributes:</h3> <table> <tbody> <tr> <td>Fix Request #</td> <td>6546482/solaris_10u5</td> </tr> <tr> <td>Target</td> <td>solaris_10u5</td> </tr> <tr> <td>Customer Status</td> <td>10-Fix Delivered</td> </tr> <tr> <td>Severity</td> <td>3-Medium</td> </tr> <tr> <td>Duplicate Of</td> <td></td> </tr> <tr> <td>Committed Version</td> <td>s10u5_03</td> </tr> <tr> <td>Fixed Version</td> <td>s10u5_03</td> </tr> <tr> <td>Integrated Version</td> <td>s10u5_03</td> </tr> <tr> <td>Verified Version</td> <td></td> </tr> <tr> <td>Last Updated</td> <td>2010-04-02 00:00:00 GMT+00:00</td> </tr> </tbody> </table> <br> <h3>Suggested Fix:</h3> See Main CR <table> <tbody> <tr> <td>Date Modified</td> <td>2007-10-10 00:00:00 GMT+00:00</td> </tr> </tbody> </table> <br> <p> </p>