Release Notes : LineRate Release Notes, version 2.6.0

Applies To:

Show Versions Show Versions

LineRate

  • 2.6.0
Release Notes
Original Publication Date: 05/14/2015 Updated Date: 04/18/2019

Summary:

This release note documents LineRate version 2.6.0.

Contents:

New Features and Enhancements

LRS-19058 Expanded cloud deployments in Amazon EC2

In version 2.6.0, LineRate can be deployed independently in Amazon EC2 or can be used for cloud bursting into the Amazon EC2 cloud. Details about how to use LineRate in Amazon EC2 can be found at
https://docs.lineratesystems.com/087Release_2.6/100Getting_Started_Guide/120Installing_Proxy/Using_LineRate_in_Amazon_EC2.

LRS-27191 DNS support for LineRate scripts

LineRate Scripting API now supports DNS API similar to the Node.js DNS API. The support for DNS API enables the scripts to make DNS queries as well as to initiate off-box connections using DNS Names.

The details of the LineRate Scripting DNS API can be found at
https://docs.lineratesystems.com/087Release_2.6/450Scripting_API_Reference_Guide/DNS.

LRS-30493 Improved integration with orchestration systems with HTTP Basic authentication

This release improves integration of REST API into various orchestration tools with support for HTTP basic authentication. In addition to HTTP cookie-based authentication, LineRate REST API can be accessed using HTTP Basic authentication. For more details on this feature, check out the REST API Reference guide at https://docs.lineratesystems.com/087Release_2.6/250REST_API_Reference_Guide/000Getting_Started_with_the_REST_API/200Accessing_the_REST_Server.


Resolved Issues

LRS-31641 (Severe) backup and restore commands can fail occasionally and may cause the system to generate core files

Symptoms:
The following messages are seen in the systems logs:
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
what(): accept: Bad file descriptor
This may be accompanied by the system crashing and generating a corefile for the controller process.

Conditions:
Running a backup or restore command from an scp URI.

Workaround:
Point the backup or restore to a local file instead of an scp URI. The files can subsequently scp'ed from the system using the bash shell.

Additional Information:

LRS-32595 (Severe) Real server health is incorrectly reported as "?" (that is, unknown)

Symptoms:
In the output from "show real-server brief", real-server health is incorrectly reported as "?".

Conditions:
You have a real server with a health monitor attached to it, and they both have an admin-status of "online". Then, you execute this series of steps in quick succession:
1. delete the health monitor.
2. delete the real server.
3. re-create the real server (using the exact same name).
4. change the real server admin status to "online".
5. re-create the health monitor (using the exact same name).
6. re-attach the health monitor to the real server.

After this, the real server health is reported as "?" (that it, unknown), instead of "up (mon off)".

Workaround:
If you bring the health monitor online, the real server health will change from "?". Also, if you were to wait for a second or two between the time you delete the real server and re-create it (or if you were to use a different real server name), you won't encounter this error condition.

Additional Information:

LRS-36507 (Severe) Running a script with timers causes a core dump in some cases

Symptoms:
The system logs in controller.messages states that an lb_http process exited due to "signal 10 (core dumped)" upon execution of a method in scripting.

Conditions:
Running a script which has the following characteristics:

  • Script sets a timer with a callback function to be invoked when a timeout occurs.
  • In the timer callback function, the script clears the timer, and then runs some code that triggers an uncaught exception.

Workaround:
If clearing a timer in a timer callback function, ensure there are no uncaught exceptions in the rest of that function.

Additional Information:

LRS-33231 (Moderate) IpFilter rules created via REST cannot be deleted via CLI

Symptoms:
Some IP filters cannot be removed via CLI commands.

Conditions:
If an IP filter has been created through the GUI or REST API, the required precision on the priority field may not be included. If an IP filter rule is created via the GUI or REST without the necessary decimal precision, these rules then can only be matched and removed via the GUI or REST and not through the CLI.

Workaround:
Use the REST API to remove offending IP filter rules. Manual curl example:
If a rule of "1.0 10.0.0.10 32" was added to filter list "fl-01" and cannot be removed using the CLI, this series of REST commands will work.
$ curl -k -c cookie.jar -d "username=<user>&password=<password>" https://<linerate_IP>:8443/login
$ curl -k -X DELETE -b cookie.jar -d "@ipfilter.json" "http://<linerate_IP>/lrs/api/v1.0/config/ip/filterList/fl-01/1.0%2010.0.0.10%2032" -H "Content-Type: application/json"

Where the file ipfilter.json contains:

{ "type": "uint32", "data": 1, "default": false }

Additional Information:

LRS-35569 (Moderate) SSL protocol disable to empty string does not warn

Symptoms:
Modifying an SSL profile to contain an empty protocol disable list does not produce a warning about enabling insecure protocols SSLv2 and SSLv3. The insecure protocols do not actually become enabled, but appear to be when viewing the SSL profile.

Conditions:
Set an SSL profile to contain an empty protocol disable list using: protocol-disable-list ""

Workaround:
None known.

Additional Information:
The SSLv2 and SSLv3 protocols are disabled by default. Enabling SSLv2 and/or SSLv3 is possible if at least one protocol is disabled, but setting the disabled list to an empty string resets the default behavior of SSLv2 and SSLv3 as disabled.

LRS-37411 (Moderate) REST API ignores Content-Type header for PUT and POST requests

Symptoms:
Content-Type header on REST API requests ignored and always treated as JSON.

Conditions:
Make a PUT or POST HTTP request to the LineRate REST API for URLs in the "/lrs" hierarchy with the Content-Type header set to something other than application/json.

Workaround:
Use application/json Content-Type header when making requests to the LineRate REST API.

Additional Information:

LRS-43226 (Moderate) nd6_dad_timer messages seen in syslogs

Symptoms:
Message "nd6_dad_timer: called with non-tentative address" seen in syslogs.

Conditions:
IPv6 is enabled on an interface.

Workaround:
None known

Additional Information:
This message has no impact on the functionality of the system.

LRS-44796 (Moderate) Unable to PXE boot LineRate guest with 2 CPUs on KVM hypervisor

Symptoms:
LineRate hangs during boot.

Conditions:
The issue happens when running LineRate as a guest under the KVM hypervisor, assigning the guest 2 vCPUs and using Virtio NICs.

Workaround:
Give the guest 4 vCPUs or switch to virtualized Intel E1000 NICs.

Additional Information:

LRS-47865 (Moderate) System does not accept client requests for IPv6 addresses

Symptoms:
System does not accept client requests.

Conditions:
Virtual IP interface is assigned to an IPv6 address.

Workaround:
Disable and re-enable the virtual IP interface via "admin-status offline" then "admin-status online".

Additional Information:
This is believed to occur when the virtual IP interface is created immediately after applying an IPv6 address to an interface. This can happen in an automated environment when the configuration is applied programmatically, or when applying the startup-config after a system reboot.


Open Caveats

LRS-26006 (Catastrophic) Man-in-the-middle vulnerability when using OpenSSL with weak ciphers (CVE-2014-0224)

CVE
CVE-2014-0224: An attacker using a carefully crafted handshake can force the use of weak keying material in OpenSSL SSL/TLS clients and servers.

Conditions:
Vulnerable version of OpenSSL is on the system and weak ciphers are enabled.

Workaround:
Disable known weak ciphers.

Additional Information:
Systems using OpenSSL 0.9.8 pre "za", OpenSSL 1.0.0 pre "m", or OpenSSL 1.0.1 pre "h" are vulnerable. LineRate has been updated to a more recent version of OpenSSL.
More details on this vulnerability can be found at http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0224

LRS-28585 (Catastrophic) Crash under high load when accessing REST API or GUI

Symptoms:
System stops processing traffic and becomes, at least temporarily, unusable via SSH. System may reboot.

Conditions:
System is under high load and a configuration change is made via the REST API.

Workaround:
None known.

LRS-34945 (Catastrophic) lb_http dumps core during garbage collection if script has dropped due to error and not timeout

Symptoms:
If a script catastrophically errors after startup, the ensuing garbage collection pass may cause the lb_http to drop core.

Conditions:
A script must be configured, compilable, and set to "admin-status online".
The script's status shows as "DOWN: run-time error" and a "Last Error" field is included in the output ("show script <name>" command).

Workaround:
Correct the error reported in the Last Error field. This field will display a detailed error report. Correct the script error to run normally.

Additional Information:

LRS-272 (Severe) Real servers should not allow max-connections of more than 64,511

Symptoms:
Load balancer stops passing traffic and continually tries to open a connection to a single real server. Statistics show Other connection errors climbing rapidly for one real server.

Conditions:
A real server is configured with max-connections as 0 (unlimited) or a number larger than 65536 - 1024 = 64512.

Workaround:
Set max-connections to a number between 0 and 64511.

Additional Information:
The bug is triggered because a single source IP address can have at most 65535 connections open to a single destination IP address. Since the LineRate Proxy will not use source ports less than 1024, the limit is 64511.

LRS-543 (Severe) Error when changing a next hop route to a direct route

Symptoms:
The 'ip route' command gives an error when attempting to change a direct connect to a gateway route or vice versa. Error is:
ERROR: Unable to create static route: Route already exists with target of <Interface or IP>

Conditions:
An IP route to a destination network is already configured and you wish to change the route.

Workaround:
Remove the route with 'no ip route' and then add the new target

Example showing issue:
host-1-16(config)# ip route 2.0.0.0/8 bce1
host-1-16(config)# ip route 2.0.0.0/8 10.201.128.115
ERROR: Unable to create static route: Route already exists with target of bce1

LRS-722 (Severe) Ethernet link saturation may cause unintentional CARP failover

Symptom:
CARP failover happens even though the master device is healthy and operating normally.

Conditions:
Passing traffic at or near full link capacity on the same interface as CARP is configured.

Workaround:
Manage traffic flow to an HA cluster such that peak traffic load is not greater than approximately 90-95% of link capacity.

Additional Information:
When traffic on a particular Ethernet link approaches the maximum capacity for that link, outgoing packets on the link will be dropped. Currently, the system does not distinguish CARP packets from any other traffic on the link. If too many CARP packets get dropped, one of the standby units in the HA cluster will notice that the master unit is no longer advertising and take over the master role.

LRS-733 (Severe) ix interfaces with no cable sometimes show link up

Symptoms:
Some ix interfaces (Intel 82598 and 82599) will have link state change periodically when no physical link is present.

Conditions:
Idle disconnected interfaces.

Workaround:
None. This is a benign side effect associated with interrupts on disconnected interfaces.

LRS-762 (Severe) interface shutdown should bring link down

Symptom:
When "shutdown" is configured on an ix or igb interface, the Ethernet link remains active, although no traffic passes.

Conditions:
A device with ix or igb interfaces.

Workaround:
None.

Additional Information:
When these Ethernet interfaces are configured with "shutdown", although the link carrier remains active, no packets are received or transmitted.

LRS-943 (Severe) Shell may become slow under traffic load

Symptoms:
LROS shell responds slowly to commands.

Conditions:
System is operating at very high CPU due to traffic processing.

Workaround:
None

LRS-1032 (Severe) Turning off tcp-multiplex with no real-server idle-timeout and low max-connection count may cause connection hang

Symptom:
After turning tcp-multiplex off within a virtual-server, incoming client HTTP requests may stop being completed.

Conditions:
The real-server must have max-connections set to a low number (a number near the number of load balancing processes, typically near 16) and idle-timeout must not be set on the real-server and tcp-multiplex must be enabled on the corresponding virtual-server. Then, after tcp-multiplex is disabled from the virtual-server, the symptoms may occur.

Workaround:
Always set idle-timeout on real-servers. Or, if you experience this problem, first disable tcp-multiplex, then manually force each real-server associated with that virtual-server to "admin-status offline" then back to "admin-status online".

LRS-1081 (Severe) Copy/Paste of keys and certificates are inserted in running config incorrectly

Symptom:
Copy and paste of SSL certificates and keys are not inserted into the configuration correctly.

Condition:
System configured for SSL profiles.

Workaround:
Users can put the certificate and key configuration in a file and use the lros_shell --config option in bash to enter the configuration.

LRS-1271 (Severe) Multiple nodes in failover group may become master if link comes up before switch begins forwarding packets

Symptoms:
Multiple nodes in a failover group show that they are master for the failover group.

Conditions:
The switch that the LROS system is attached to has a delay between when link is established on a port and when the port is ready to RX/TX packets. This delay is typical in many switches and is related to spanning tree features.

Workaround:
Turn off spanning tree features on the switch port directly connected to each LROS system. On Cisco switches, the command to disable spanning tree for a port is "spanning-tree portfast". Refer to your switch documentation for more information.

LRS-1447 (Severe) In non tcpmux mode HTTP CONNECT message is forwarded to real-server even when "is-proxy" http option for the real-server is not set.

Symptoms:
HTTP CONNECT request is forwarded to the real server instead of being blocked

Conditions:

  • The virtual server has tcp-multiplex disabled.
  • The virtual server has the http protocol option "forward-connect-request" enabled.
  • For all the real servers attached to the virtual server, the http protocol option "is-proxy" is disabled.

Workaround:
Use tcp-multiplex for virtual servers.

Additional Information:
Under the above specified conditions a HTTP CONNECT request is forwarded to the real-server incorrectly. If the real server is indeed a proxy, a proper response may be received. In addition, when this happens, the http connect specific statistics would not show proper values. Specifically, the "httpServerConnectRequests" stat may not be updated properly.

LRS-1498 (Severe) Internally generated HTTP 502 responses do not always increment counter

Symptom:
No statistics are updated under certain conditions when the load balancer returns a 502 Bad Gateway message.

Condition:
Particular error cases within the load balancer do not update the internally generated 502 response error counters.

Workaround:
None known.

LRS-1590 (Severe) Certificate bundles without an END line for each BEGIN are accepted

Symptom:
Certificate bundles that are missing an END CERTIFICATE are accepted by the command line.

Condition:
System configured for certificate bundles and a bundle that is missing an END statement is entered into the configuration.

Workaround:
None. Use care when entering the certificates and verify that all certificates the system has are correct using the show certificate brief command.

LRS-1764 (Severe) Stripping certain headers may result in unexpected behavior

Symptom:
HTTP connections between client and virtual server are dropped, corrupted, etc.

Conditions:
Virtual server has the following configuration:
protocol http
strip-request-header <STRING>

where <STRING> contains certain special keywords, including but not necessarily limited to, the following special keywords that affect the core load balancing functionality:
Transfer-Encoding
Content-Length
Content-Type
Connection
Host

Workaround:
None known.

LRS-1835 (Severe) CARP: group status goes admin down when changing interface MTU

Symptom:
The "show carp" command shows one or more CARP groups as administratively down, even though the groups are configured to be administratively up (not shutdown).

Conditions:
The MTU configuration on the parent interface of the CARP group was changed.

Workaround:
In configuration mode for the parent interface of the CARP group, perform a "carp <group> shutdown" followed by "no carp <group> shutdown".

LRS-1927 (Severe) lb_http process stuck at 100% CPU

Symptoms:
A load balancer data path process (process name lb_http) runs at 100% CPU, even though network traffic may not be at a high level.

Conditions:
System configured for load balancing or perhaps SSL offload. Other conditions uncertain.

Workaround:
Killing the lb_http process appears to resolve the high CPU issue. The system will restart the process.

LRS-1976 (Severe) System may become unresponsive when changing number of load balancer processes while under traffic load

Symptoms:
The system shell may become unresponsive while configuring "load-balancer processes <nprocs>" (version 1.6.x) or "proxy processes <nprocs>" (version 2.x) and may begin to send TCP reset packets in response to new incoming traffic.

Conditions:
Changing the number of load balancer or proxy processes while under very high traffic load.

Workaround:
Note: Consider your changes carefully before using this workaround, because this command can have a serious impact on your system. For information about the command discussed below. For 1.6.x, see https://docs.lineratesystems.com/100Release_1.6/200CLI_Reference_Guide/Configure_Commands/Load_Balancer_Mode_Commands. For 2.x, see https://docs.lineratesystems.com/200Release_2.0/200CLI_Reference_Guide/Configure_Commands/Proxy_Mode_Commands.

Configure the number of load balancer or proxy processes, or reduce traffic load before changing this configuration option.

LRS-2082 (Severe) System closes/opens an excessive number of real-server connections with tcp-multiplexing enabled

Symptoms:
The system opens and closes more connections to real servers than is necessary.

Conditions:
The system is configured for load balancing with tcp-multiplex enabled on one or more virtual servers.

Workaround:
None known.

LRS-2151 (Severe) Load balancer might reissue a request that was partially transmitted

Symptoms:
Web servers see duplicate request header fragments that were not issued multiple times by actual clients.

Conditions:
This occurs when max-in-flight is greater than 1 on a real server, that real server prematurely closes a connection during a response, and a request header is only partially transmitted.

Workaround:
Set max-in-flight to 1 on all real servers.

LRS-2215 (Severe) Session caching doesn't work with TLSv1 and some SSL servers

Symptoms:
SSL session cache on a remote SSL server is not being used, even though LineRate Proxy is configured to use SSL caching when initiating sessions to the remote server. This was seen when using a Spirent test product as the SSL server, but could be present with other SSL implementations as well.

Conditions:
System is configured for SSL initiation to real servers and is configured to use TLSv1 (which is included in the default cipher list) and is configured to use session caching.

Workaround:
None known.

LRS-2217 (Severe) Load balancer processes consume all available memory

Symptom:
Load balancer datapath processes consume a large amount of memory and may consume all free memory. This may also result in processes being killed, which can be seen in the system logs as messages similar to:
LROS kernel: <3>pid 1744 (lb_http), uid 0, was killed: out of swap space
LROS: ProcManager: lb_http exited: exited due to signal 9. Restarting.

Conditions:
Load balancer is configured for SSL and is processing a large number of SSL negotiations simultaneously, in excess of the platform's SSL capabilities. This happens more readily at larger SSL key sizes, particularly with 4096-bit keys.

Workaround:
Configure the system to avoid having too many active SSL connections at any given time by adjusting the max-client-conn setting. In version 1.6.x, use load-balancer max-client-conns <conns>. In version 2.x, use proxy max-client-conns <conns>. For a device with 24 GB of memory, a setting of 200000 should be sufficient.

LRS-2221 (Severe) Virtual server has huge queue without traffic

Symptoms:
The virtual server request manager queue contains a large number of requests, which may result in the load balancer sending HTTP 503 responses if the queue grows larger than 10,000 entries.

Conditions:
The system is configured for load balancing and a virtual server has real servers attached to it, but one or more of the real servers is not responding to requests and the non-responsive real server does not have a health monitor enabled to check its health.

Workaround:
Configure a health monitor for all real servers and enable the health monitor.

LRS-2419 (Severe) Installer: ISO install fails on some HP machines over virtual CD/DVD

Symptom:
Installation from ISO image via virtual DVD fails due to mount failure of cd9960 device.

Conditions:
This issue has only been observed on HP DL120 G7 servers with BIOS version J01 from 05/20/2011 and ILO firmware version 1.26 from 8/26/2011. This BIOS version is not the official supported version of BIOS from HP.

Workaround:
Install LROS from USB key or physical optical drive.

Additional information:
On the same servers, BIOS version J01 from 4/21/2011 and ILO version 1.20 from 3/14/2011 work correctly with virtual DVD install.

LRS-2588 (Severe) Real server latencies may not be accurate

Symptoms:
Real server latency numbers, observed with "show real-server <name> stat" may vary quickly and widely in a short period of time.

Conditions:
System is configured for load balancing and has active traffic to real servers.

Workaround:
None known

LRS-2603 (Severe) Duplicate IP addresses can be configured on multiple interfaces

Symptom:
The same IP address can be configured on more than one interface. Virtual IPs may reset traffic after.

Condition:
LROS 1.4.0

Workaround:
None, user must ensure that IP addresses are not duplicated in the configuration

LRS-2628 (Severe) Real server continues to receive traffic after removal from group

Symptoms:
A virtual-server contains a real-server group and that group contains a real server that is subsequently removed from the group. The removed real server continues to take traffic from the virtual-server, even though it is no longer configured to do so.

Conditions:
System is configured for load balancing.

Workaround:
Once in this condition, using 'admin-status offline' followed by 'admin-status online' on the affected real-servers resolves the issue.

LRS-2685 (Severe) Assertion failure when increasing number of lb_http processes under load

Symptoms:
Logs report that an lb_http process has exited with signal 6.

Conditions:
The configuration was just changed to increase the number of load balancer processes.

Workaround:
Remove load from the proxy before changing number of processes.

Additional Information:
While an lb_http process will crash, it will immediately restart (usually in under 1 millisecond). Only connections to the crashing processes will be lost.

LRS-2727 (Severe) Weighted Round-robin free load-balancing algorithm results in a 10% performance drop

Symptoms:
Performance dips by about 10% on a fully loaded system when a virtual server's load balancing algorithm is weighted-round-robin-free (vs. round-robin-free)

Conditions:
The LineRate Proxy is at maximum load.

Workaround:
With equally weighted servers, use round-robin free instead.

Additional Information:
The 10% performance drop only occurs at near maximum load. The drop in performance will be influenced by the percentage of requests load balanced by the weighted round robin free algorithm. If most of the traffic goes to a virtual server with a different algorithm, no performance drop may be observed.

LRS-2797 (Severe) REST not working correctly when configured to use the same IP address and port as a virtual-ip interface

Symptom:
REST server not handling requests properly.

Conditions:
Configure the REST server to accept connections on the same IP address and port as that of a virtual IP.

Workaround:
This configuration is not supported. You must configure REST server to use either a different address or port.

LRS-2860 (Severe) Unavailable servers cause unbounded reissue attempts for requests

Symptoms:
A virtual server has a rapidly growing request queue, even with minimal load.

Conditions:
All real servers for a given virtual server have no health monitor configured, and these real servers refuse all connections from LineRate Proxy.

Workaround:
Add health monitoring to the real servers.

Additional Information:
The queuing is caused by LineRate Proxy constantly reissuing requests when it fails to connect to a real server, instead of eventually giving up and returning an error response.

LRS-2897 (Severe) System timer threads may in aggregate deny service to management tasks

Symptoms:
Management console becomes unresponsive under certain load conditions.

Conditions:
Certain load patterns that generate an excessive amount of deferred work handled by multiple system-level timer threads (tmr_sys, tmr_tcp, tmr_syn, etc.) such that two or more of these high-priority threads consume sufficient management CPU resources as to deny service to the interactive management tasks. The load on these threads may be observed by running the following command, 'top -S 1000 | grep swi4'.

Workaround:
Note: Consider your changes carefully before using this workaround, because these commands can have a serious impact on your system. For 1.6.x, see https://docs.lineratesystems.com/100Release_1.6/200CLI_Reference_Guide/Configure_Commands/Load_Balancer_Mode_Commands. For 2.x, see https://docs.lineratesystems.com/200Release_2.0/200CLI_Reference_Guide/Configure_Commands/Proxy_Mode_Commands. Also see, https://docs.lineratesystems.com/200Release_2.0/200CLI_Reference_Guide/Configure_Commands/Scheduler_Mode_Commands.

Reserve a CPU for the timer threads by reducing the number of load balancer or proxy processes by one and moving the "process-class clock" to the next to last CPU in the "scheduler cpu manual" mode.

LRS-2966 (Severe) Memory leak when using REST to manage the system

Symptom:
Slow memory leak will eventually lead to the Out-Of-Memory Killer killing random processes. The OOM can pick any user process, which may lead to a network outage as some LROS processes do not automatically recover.

Conditions:
Extended use of the REST API to monitor the system.

Workaround:
None

LRS-2971 (Severe) Timer threads (tmr_*) may run at ~95% under load and ~65% at idle

Symptoms:
System timer threads may consume excessive CPU.

Conditions:
Unknown.

Workaround:
None needed. System performance appears unaffected.

LRS-2997 (Severe) WRRF: Service TCP load balancing ratios to real-servers is not what is configured

Symptom:
The actual ratio of load balanced connections to real servers is not the same as what is configured by a real server's weight.

Condition:
System configured with the weighted-round-robin-free load balancing algorithm on virtual servers.

Workaround:
None known.

LRS-3049 (Severe) The fact that config data is being reloaded should be made more obvious.

Symptoms:
When the user initiates a config data reload in the GUI, it should be more obvious that the config data is being reloaded (for example, activate the "spinner" in all possible "reload" scenarios and make it larger and/or more central). The problem is that the browser considers the page to have loaded before the GUI has actually finished reloading the config data (that is, the page load and config reload are two separate activities). When the browser's spinner or progress bar stops moving, it's easy to assume the application is finished reloading the config data - and this assumption is likely to always be incorrect.

Conditions:
The UI reloads the underlying config data in the following cases:

  • when you first log in
  • when you click the browser "reload" button
  • when you click the application "reload config data" button

Workaround:
None.

LRS-3059 (Severe) Icons should provide useful health/status info on hover/mouseover.

Symptoms:
When you hover over icons, they don't display relevant health/status information (for example, diagnosis of the underlying problem).

Conditions:
Configuration contains objects that are "down", misconfigured, or only partially configured.

Workaround:
None.

LRS-3074 (Severe) When the session idle timeout has been reached, the icons become broken images and the "reload" button fails to redirect the user to the login page.

Symptoms:
After clicking a tab, object icon, or configuration reload button, the GUI displays broken image icons.

Conditions:
Broken links can occur after the session idle timeout period has expired.

Workaround:
Log in to the application again by clicking the browser reload button or by clicking the log out link located at the top of the application. Enter a valid user name and password in the login page to start a new session.

LRS-3143 (Severe) Loading of large configuration fails sometimes

Symptoms:
When a large startup configuration is applied to the system via 'lros_shell --config <config-filename>", the operation fails and error messages similar to the following may be present in the system logs:
ERROR: timeout while sending remote set to applic
Connection to host-67 closed.ttp/realServer/rs-1213-02
ashok@fiji:~$ 0 host-67-data LROS: health: *Error* controller communication exception (parseHeader):
Encountered EOF before it was expected
Mar 29 10:58:20 host-67-data LROS: health: *Error* Communication failure with controller. Restarting
communications and resyncing.
Mar 29 10:58:24 host-67-data LROS: Health SysdbRemote: An operation unexpectedly timed out. Path: /conf
ig/certificates
Mar 29 10:58:59 host-67-data LROS: health: *Error* controller communication exception (parseHeader):
Connection reset by peer

Conditions:
This problem has been observed only once while loading a large configuration file (2500 Virtual-servers, 2500 virtual-ip's, 10000 real-servers) via the command 'lros_shell --config <config-filename>'.

Workaround:
Apply new configuration in smaller chunks when using 'lros_shell --config <config-filename>' command.

LRS-3189 (Severe) "Down" object rows should be highlighted appropriately (ie. even when they are not the "selected" object).

Symptoms:
The table rows for objects in the GUI that are "down" are not highlighted in any way.

Conditions:
Find the table rows for one virtual-ip object that is "down" (that is, has a "status" of "DOWN: administratively offline") and one virtual-ip that is "up" (that is, has a "status" of "up"). Note that both rows essentially look the same (that is, the "down" row is not appropriately highlighted).

Workaround:
None.

LRS-3539 (Severe) Client connections hang if TCP multiplex is enabled when real server does not have max-connections configured

Symptom:
Incoming connections from clients to a virtual IP do not close or get a response and appear to hang.

Condition:
The virtual server where the virtual IP is attached has tcp-multiplex enabled and one or more of the real servers attached to that virtual server have no max-connections value set.

Workaround:
Disable tcp-multiplex or set max-connections on all real servers where you intend to use tcp-multplex.

LRS-3676 (Severe) Upgrade command doesn't give clear error on insufficient space

Symptom:
An error message similar to the following is displayed when using the upgrade command:
ERROR: Error upgrading: Running /boot/install failed.
Unexpected error upgrading LROS

Messages similar to the following will also be present in /var/log/controller.messages:
Jun 12 13:29:41 LROS LROS: Validating the running configuration is saved...
Jun 12 13:29:41 LROS LROS: Mounting Root image (/home/linerate/backups/LROS-1.5.3-R-amd64.upg)...
Jun 12 13:29:41 LROS LROS: Validating the running configuration is saved...
Jun 12 13:29:42 LROS LROS: Mounting Root image (/home/linerate/backups/LROS-1.5.3-R-amd64.upg)...
Jun 12 13:29:42 LROS LROS: Setting undefined device to ad0
Jun 12 13:29:42 LROS LROS: Error: insufficient space on target
Jun 12 13:29:42 LROS LROS: space required : 3 G
Jun 12 13:29:42 LROS LROS: space available: 2 G

Conditions:
The system is low on disk space and there is not enough space to upgrade the image.

Workaround:
None.

LRS-3801 (Severe) Unacceptable latency with L4 load balancing

Symptom:
Very high latencies are seen when doing layer 4 (TCP) load balancing. Average latencies may be above 100ms and some latencies may approach 700ms.

Condition:
Virtual server is configured with 'service tcp' and the system CPU is above a certain threshold, about 75-80%.

Workaround:
Do not allow CPU to get to 75% by distributing traffic among more nodes in a cluster.

LRS-3802 (Severe) Error messages in log: write error: broken pipe

Symptom:
Log messages similar to the following appear in system logs:
LROS: Termination(201.0.69.1:9952 -> 201.0.13.4:8080) to Initiation(201.0.15.1:9936) write error: Broken pipe

Condition:
Unknown, but may be related to layer 4 (TCP) load balancing.

Workaround:
None known

LRS-3804 (Severe) SNMP output traffic not incrementing for port channel interface

Symptom:
SNMP counter for output byte count on a port channel interface does not increment. However, input byte count works correctly.

Condition:
System has a port channel interface configured and that interface is being monitored for output traffic via SNMP IF-MIB.

Workaround:
Poll the physical interface or subinterfaces, instead of the port channel, for output data counts.

LRS-3806 (Severe) Second interface in failover group lags on carp state change

Symptom:
When one CARP group within a failover group changes state, there may be a delay before the other CARP groups in the failover group also change state.

Conditions:
System is configured with a failover group that has multiple CARP groups configured within the failover group.

Workaround:
None known.

LRS-3808 (Severe) SNMP queries slow under traffic load

Symptom:
SNMP queries to a system may slow down as the CPU and traffic load increase on the system.

Condition:
CPU load on the system is somewhat high.

Workaround:
Apply the following configuration, where <mgmtCPU> is the number of the highest CPU on your system. That number is 15 for a system with 16 hyperthreads or 23 for a system with 24 hyperthreads.

scheduler cpu manual
process-class regex "snmpd" "<mgmtCPU>"

LRS-3810 (Severe) Performance drop when doing tcpdump

Symptom:
Overall system performance may degrade more than necessary when performing a packet capture using the tcpdump utility.

Conditions:
Packet capture is being performed on a VLAN sub-interface which is attached to a port channel.

Workaround:
Ensure traffic load is fairly low before performing packet capture or perform the capture on a physical interface.

LRS-4065 (Severe) HTTP re-issues and 502 errors on low traffic rate virtual-servers

Symptom:
502 error HTTP responses are seen in response to some HTTP requests and the Reissues Dropped counter is incrementing.

Conditions:
1. The virtual server is configured with service http.
2. The virtual server is configured to enable tcp-multiplex.
3. There is a fairly low rate of traffic on the virtual server.
4. The servers attached to the virtual server have timeouts set that close connections that are opened but where no request is sent on that connection (this is often called connection timeout).
5. The real server configuration on the system has a keepalive-timeout that is longer than the timeout from #4 above or does not have keepalive-timeout set.

Workaround:
Configure the system's real server keepalive-timeout to a value that is less than the server's timeout for connections that do not issue a request.

LRS-4498 (Severe) Intermittent carp problems on systems with bce interfaces

Symptom:
CARP peers both switch to master and are unable to contact each other, leading to data traffic problems.

Conditions:
CARP is configured on a bce interface.

Workaround:
Install or replace bce interfaces with another vendor type, such as igb, ix, or em.

LRS-5884 (Severe) Systems with oce interfaces may stop processing traffic briefly when receiving fragmented IP packets

Symptom:
Traffic coming through an oce interface may stop briefly when that interface is receiving fragmented IP packets with certain characteristics

Conditions:
Fragmented IP packets are being received on the interface.

Workaround:
None known

LRS-6631 (Severe) Only a subset of proxy processes handle new client connections

Symptoms:
Only a subset of data path proxy (lb_http) processes handle new client connections to a virtual IP. You might see this reflected in CPU utilization.

Conditions:
If the sock_mgr process has crashed (as evidenced by a message in the system logs that "sock_mgr" process has crashed and restarted, or a core file in the /var/crash directory is created) and:

  • a virtual IP is deleted and re-created with the same IP address and port or
  • a virtual IP is deleted and another virtual IP is created with the IP address and port of the deleted virtual IP or
  • a virtual IP is admin offline/onlined or
  • a proxy (lb_http) process restarts or
  • the number of proxy processes in the system is increased.

Workaround:
Reduce the proxy processes to 0 and then set it back to previous value.
Use the following config command to set the proxy processes to 0

  • proxy processes 0

Use the following to set the number of proxy processes to auto

  • proxy processes auto

LRS-6634 (Severe) After SYN flood attack, cannot enter configuration mode to configure objects for a period of time

Symptom:
During and after a SYN flood attack, you cannot enter configuration mode to modify the objects in the configuration.

Condition:
System under SYN flood attack.

Workaround:
None, after the attack is mitigated, you will be able to enter configuration mode.

Additional Information:
During this time, you may see the following messages in the /var/log/messages file:

Jan 31 11:29:17 host-71-data LROS: HTTP LBMgmt-SysdbRemote SysdbRemote ERROR: timeout while sending remote set to application. Path: /config/app/proxy/virtualIP/VIP-1
Jan 31 11:29:17 host-71-data LROS: lb_http: *Error* controller communication exception (parseHeader): Encountered EOF before it was expected
Jan 31 11:29:17 host-71-data LROS: lb_http: *Error* Communication failure with controller. Restarting communications and resyncing.

LRS-6638 (Severe) SYN flood on a single VIP with traffic causes client errors

Symptom:
During SYN flood attack on a single virtual IP, traffic to real clients is severely degraded.

Condition:
System under SYN flood attack on a virtual IP that is passing traffic.

Workaround:
None.

LRS-7380 (Severe) System needs to detect and take action on data path processes that are unresponsive

Symptom:
In some rare instances, the system can get into a state where configuration
information is not pushed to all data path processes. This can manifest
itself in different ways. For example, while in this state, if a script is
configured and turned administratively online, it may not run on all data path processes.

Condition:
This can happen when certain load balancer processes are unresponsive to
configuration commands - for example - if a bad running script got into
an infinite loop.

Workaround:
None known.

LRS-8978 (Severe) Memory leak when adding and removing large configuration

Symptoms:
System consumes more memory after removing and re-adding the same configuration.

Conditions:
System is configured for load balancing with virtual servers, real servers, and virtual IPs.

Workaround:
None.

Additional Information:

LRS-9373 (Severe) Forward proxy configuration which includes LROS interface IP in IP range (or specifies it as virtual IP) is vulnerable to DoS attack

Symptoms:
CPU and memory usage increase until lb_http processes crash.

Conditions:
A forward proxy configuration exists which contains a system IP address, and a request from a client to this IP address is made.

Workaround:
Rewrite forward proxy definitions to use multiple ranges, excluding the system IP addresses.

Additional Information:

LRS-9608 (Severe) Memory leak when script in auto-restart mode has runtime error

Symptoms:

Memory usage of data path processes continually increases. Depending on the size of system memory, rate of increase, system load, and garbage collection frequency, this could conceivably result in a crash due to lack of memory. If the system is not already low on memory, usage will likely plateau as the garbage collector does its job.

Conditions:

A script is repeatedly crashing with a runtime error, for example, due to a missing module.

Workaround:

Set the script to manual restart mode using:
conf
script <scriptname>
restart-mode manual

LRS-9709 (Severe) LROS usb image no longer fits on a 2GB thumb drive.

Symptoms:
LROS usb image no longer fits on a 2GB thumb drive.

Conditions:
The introduction of the scripting feature set caused the LROS usb image to increase in size (from just under 2GB to just under 4GB).

Workaround:
Use 4GB media instead of 2GB media.

Additional Information:

LRS-10248 (Severe) Zombie connections not reaped for VIP using IP other than CARP IP

Symptoms:
System has stale TCP connections after switching from CARP master to backup role.

Conditions:
The virtual IP address is configured to use the IP address of a loopback interface, to which packets are internally routed from a public-facing interface, such as physical or VLAN.

Workaround:
None. TCP connections will either time out and expire, or if the unit becomes master again, incoming TCP connections will trigger collisions upon which the system will respond with a TCP reset.

Additional Information:

LRS-13156 (Severe) Panic during IPv6 multicast group subscription teardown from third-party application

Symptoms:
Kernel panic can occur when a third-party application leaves an IPv6 multicast group.

Conditions:
In some cases, a third-party application that leaves an IPv6 multicast group after the group network interface is destroyed can cause a system panic.

Workaround:
User is advised to avoid installing or enabling applications or services that join IPv6 multicast groups.

Additional Information:
Installation of additional software on the LineRate Proxy hardware is not supported.

LRS-13202 (Severe) Scripts can set HTTP/1.1 headers on HTTP/1.0 responses, like node

Symptoms: An HTTP/1.0 client receives a response with HTTP/1.1 headers and behavior when a script sets HTTP/1.1 headers.

Conditions: A script must actively set the 'Transfer-Encoding' header:

vs.on('request', function(req, res, next) {
  // BAD: Didn't check req.httpVersion first!
  res.setHeader('Transfer-Encoding', 'chunked');
});

Workaround: Do not set the 'Transfer-Encoding' header yourself. Scripting will automatically add it if necessary. If the script needs to set the 'Transfer-Encoding' header, it should first check the request HTTP version:

vs.on('request', function(req, res, next) {
  if (req.httpVersion === '1.1') {
    // This isn't necessary; scripting will set it for you.
    res.setHeader('Transfer-Encoding', 'chunked');
  }
});

Additional Information: This is the same behavior as node.js.

LRS-14080 (Severe) System generates core file when script incorrectly pipes and writes the response

Symptoms:
A client request terminates early, because the data path process has crashed. The system also generates a core file.

Conditions:
A script is intercepting the response, the script incorrectly pipes and writes to the response.

Workaround:
It is invalid to pipe and write to the response. A script should either pipe or write to the response, not both.

Additional Information:

LRS-14081 (Severe) System is slow and lb_http processes are at 100% for some time

Symptoms:
System becomes slow and lb_http processes are shown at 100% CPU even though no traffic is active on the system.

Conditions:
System with a large number of open stale connections.

Workaround:
None known. The system will eventually clear all of the stale connections and return to a normal state.

Additional Information:
This happens on a system that has had a large number of open connections that have been terminated abnormally (no RST or FIN), and then more connections using those same ports have been sent causing the device to reset those connections.

LRS-17539 (Severe) Data path processes are terminated due to out-of-memory due to a script consuming all memory with HTTP.ClientRequest

Symptoms:
Data path processes are terminated due to out-of-memory.

Conditions:
A script has hundreds of thousands or millions of outstanding HTTP.ClientRequests and HTTP.ServerRequests.

Workaround:
The proxy can be configured to limit the admission of new traffic using the "proxy max-client-conns" command. The script can be written to retire requests quickly even if the HTTP.ClientRequests that it makes are not satisfied.

Additional Information: Every time a script makes an HTTP.ClientRequest, memory is allocated that can't be freed until the response arrives or times out. If the HTTP.ClientRequest must be completed before an outstanding HTTP.ServerRequest/HTTP.ServerResponse pair can be completed (that is, a request from the client of the proxy), consider setting a low timeout on the pair to ensure memory is reclaimed quickly. At scale, try to minimize the amount of data and total transactions that must be "in flight" in the proxy at a given time.

LRS-18267 (Severe) "ERROR: Unable to create socket: Interrupted system call" message when creating a virtual IP

Symptoms:
When creating a virtual IP, the error message "ERROR: Unable to create socket: Interrupted system call" is seen.

Conditions:
The system is configured with the interfaces bound together using LACP. The IP address of the LACP interface is being used for the IP address of the virtual IP.

Workaround:
None known.

Additional Information:

LRS-18885 (Severe) Connections do not appear on the expected VIP when using IP address range

Symptoms:
Connections do not appear on the expected VIP when using IP address ranges.

Conditions:
Configure the system with two or more virtual IPs that use IP address ranges. The IP address ranges on the virtual IPs must overlap.

Workaround:
Adjust the VIP ranges so they do not overlap.

Additional Information:

LRS-20742 (Severe) Management commands on very small-scale virtual deployments take a long time

Symptoms:
Management commands via SSH or REST query take a long time (10's of seconds to minutes) to complete.

Conditions:
LineRate is deployed on a physical machine or as a virtual guest with 2 CPUs. This means two total processing cores (or hyperthreads) available to LineRate, not two CPUs each with two or more cores.

Workaround:
Configure request rate limiting on all virtual servers, subcommand of virtual server service http, appropriate for the features configured in your environment.

Additional Information:
On a system with two CPUs, the management process is located on the same CPU as the data path process. Under heavy traffic load, the data path process will consume 100% of the CPU, starving the management process. Management access to the system will return to normal as traffic load dissipates.

LRS-32193 (Severe) CVE-2014-3513 Possible memory exhaustion when processing malicious SSL handshakes with SRTP extension

CVE:
CVE-2014-3513 (openssl): Memory leak in d1_srtp.c in the DTLS SRTP extension in OpenSSL 1.0.1 before 1.0.1j allows remote attackers to cause a denial of service (memory consumption) via a crafted handshake message.

Conditions:
LineRate configured to terminate SSL connections

Workaround:
None known

Additional Information:
https://www.openssl.org/news/secadv_20141015.txt
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3513

LRS-33213 (Severe) CVE-2014-8475 (openssh + kerberos)

CVE:
CVE-2014-8475 (openssh) Possible denial-of-service attack can prevent the system from accepting any SSH connections.

Conditions:
None known

Workaround:
None known

Additional Information:
https://www.freebsd.org/security/advisories/FreeBSD-SA-14:24.sshd.asc

LRS-40732 (Severe) Configuring a user password with an exclamation point (!) comments out the rest of the password

Symptoms:
User passwords containing an exclamation points and not enclosed in quotes are truncated.

Conditions:
User passwords configured by the CLI containing exclamation points and not enclosed in quotes. For example:

username admin secret awesome!password

The admin password is set to "awesome".

Workaround:
Enclose passwords in quotes during configuration. Example:

username admin secret "awesome!password"

Additional Information:

LRS-46490 (Severe) After a forced shutdown, LineRate system will not boot

Symptoms:
System does not boot after an unclean shutdown. The console displays:

Can't load 'kernel'

Type '?' for a list of commands, help for more detailed help
OK

Conditions:
The system crashed or did not shut down cleanly shortly after issuing a "write" command.

Workaround:
At the OK prompt, type:
ls boot

The available versions to boot will be listed, along with some other files. Using the correct version (2.5.0 in this example), type:

load boot/2.5.0/loader.rc
boot

Additional Information:

LRS-47730 (Severe) Unable to transfer multiple large SSL files concurrently

Symptoms:
Unable to establish more than one SSL connection.

Conditions:
Using first cipher (AES128-GCM-SHA256) in the default cipher list for an SSL profile pegs the CPU of one lb_http process at 100%. After this occurs, client is unable to initiate additional requests through an SSL connection.

Workaround:
Non-GCM ciphers, such as AES256-SHA1, do not peg the CPU at 100%, which allows other SSL connections to be established and start file transfer via GET.

Additional Information:

LRS-110 (Moderate) In TCP Multiplex mode, a server connection may be closed even when the client is the cause of the timeout

Symptoms:
Server connections are closed when it takes longer than the response timeout for the client to receive the request.

Conditions:
A virtual server is configured with tcp-multiplex, a server is responding to a client, and the client takes too long to receive the request, the load balancer will sometimes close the server connection as well.

Workaround:
None.

LRS-183 (Moderate) Server does not always timeout properly

Symptoms:
Client connections see HTTP 502 errors, and the server statistics report reset errors.

Conditions:
Seen on some occasions when a client closes a connection while the server is responding.

Workaround:
None, however, when a new request is initiated to the server, the idle timeout for the connection goes back into effect.

LRS-410 (Moderate) Password too long is silently not accepted

Symptoms:
Passwords longer than 128 characters are silently ignored when configured. The password does not become part of the running configuration and is not saved.

Conditions:
None.

Workaround:
Use passwords shorter than 128 characters.

LRS-477 (Moderate) Some IP traffic stats do not increment

Symptoms:
The following statistics in 'show ip traffic' do not increment:

  • bad hop count
  • security failures
  • with options

Conditions:
Any IP traffic.

Workaround:
None.

LRS-483 (Moderate) Broadcast packets may be double counted in statistics

Symptoms:
Broadcast packets may be double counted at the driver layer.

Conditions:
None.

Workaround:
None.

LRS-595 (Moderate) virtual ip names should be allowed to be keywords

Symptom:
Use of a keyword as an argument to an existing command fails. For example, "virtual-ip carp".

Condition:
Entering a CLI keyword for a command's argument is not accepted by the CLI and returns an error message.

Workaround:
Use non-keyword strings as arguments for commands.

LRS-610 (Moderate) ICMP echo doesn't show up in sent ICMP stats

Symptom:
When using the ping command from an LROS system, packets are sent and received correctly, but the counter for ICMP echos sent, as viewed by 'show ip traffic' does not increment.

Conditions:
ICMP echo packets are being generated from an LROS system.

Workaround:
None.

LRS-700 (Moderate) Load balancer does not conform to RFC 2616 handling of connection headers

Symptoms:
The load balancer forwards headers named after tokens in Connection headers received in an HTTP/1.0 (or lower version) HTTP message. This could result in strange connection behavior or failure to process a request properly on a real server.

Conditions:
An HTTP/1.0 or earlier client transmits a Connection header along with other headers that match tokens in the connection header to the load balancer.

Workaround:
None.

Additional info:
Problems due to this behavior should be rare.

LRS-871 (Moderate) Command line search & edit displays strange characters

Symptom:
Command line search with the CTRL-R command inserts invalid characters with right arrow key.

Condition:
Using the CTRL-R command to reverse search the command history.

Workaround:
None

LRS-874 (Moderate) Duplicate acks during health monitoring

Symptoms:
Duplicate tcp ACK packets may be generated for HTTP queries from the health monitor.

Conditions:
Health monitoring is enabled.

Workaround:
None.

LRS-879 (Moderate) Duplicate registration message from SNMP is seen during boot up process

Symptom:
A message similar to the following is seen in the boot logs.
net-snmp[1064]: duplicate registration (lrsSnmp/inst, lrsSnmp/inst)

Condition:
Appears to happen on every boot.

Workaround:
None known.

LRS-892 (Moderate) Real server max-connections can be set to unlimited while real-server is attached to virtual server in TCP multiplex mode

Symptoms:
Real server connections grow without bound when attached to a virtual server that is in tcp-multiplex mode.

Conditions:
After configuring a virtual server to be in tcp-multiplex mode, set the max-connections limit on a real-server to 0.

Workaround:
Ensure that all real servers used with a tcp-multiplex mode virtual server have max-connections set.

LRS-936 (Moderate) Real server connections can exceed max-connections

Symptoms:
Real servers have more active connections than specified in max-connections.

Conditions:
System is configured to have more than one load balancer process (which is the default on system with more than 4 cores). Also, the configured max-connections value does not evenly divide across the total number of load balancer processes.

Workaround:
The overage is due to rounding. For example, with 15 load balancer processes and max-connections as 500, the true number of max-connections is round(500/15)*15. Either adjust max-connections down to the next lowest or next highest multiple of the number of lb_http processes. bash "ps -auxww lb_http | grep -v -E 'grep|management' | wc -l" will show the number of load balancer processes currently configured.

LRS-947 (Moderate) Certain errors are not translated to human readable form

Symptoms:
Certain error messages are cryptic. For example: "ERROR: memberRegex rs.%2A not configured on rserverGroup foo"

Conditions:
A typed command has an error.

Workaround:
None.

LRS-984 (Moderate) 'no base <name>' commands do not check for existence of object to be removed

Symptom:
Entering an invalid base name removes the base that is configured under the virtual IP or real server.

Condition:
System configured with virtual IP or real server that use bases.

Workaround:
None.

LRS-1069 (Moderate) Two virtual IPs can be configured with the same IP address/port pair.

Symptoms:
One or more virtual IP addresses appear not to obey their parameters (such as, idle-timeout, max-connections, max-in-flight, etc.)

Conditions:
Occurs when two virtual IPs are configured with the same IP address/port combination.

Workaround:
Do not configure multiple virtual IPs with the same IP/port combination.

LRS-1088 (Moderate) Controller communication exception error during boot or when changing number of processes

Symptoms:
A message similar to the following may appear in the system log:
LROS: health: *Error* controller communication exception (parseHeader): Encountered EOF before it was expected
LROS: health: *Error* Communication failure with controller. Restarting communications and resyncing.

Conditions:
This message may be printed when any of the following occur:

  • The system is booting.
  • The number of load balancer processes changes with the 'load-balancer processes' command.
  • A load balancer or health process terminates and is restarted.

Workaround:
None.

Additional Information:
The system will continue to function normally after these error messages, as the load balancer and health processes will re-establish the communication with the controller immediately and will continue to operate normally.

LRS-1120 (Moderate) Carp: Deleting carp interface makes it so IP can't be added to carp

Symptoms:
The command carp <vhid> ip <ip address> gives:
ERROR: Unable to set IP address: Internal error: No additional information available

Conditions:
A previously configured CARP interface had the same VHID as the CARP interface that shows the symptoms.

Workaround:
Restart the system and then configure CARP, choose a different VHID, or choose a different IP address (if possible).

LRS-1150 (Moderate) ctrl-Z causes issues when used in username secret

Symptoms:
The string "^Z" may appear in command history of the username line

Conditions:
Pressing control-Z may cause strange behavior with username secrets.

Workaround:
Do not use Ctrl-Z in username secrets.

LRS-1155 (Moderate) Long usernames are accepted by LROS but can't be used as logins

Symptoms:
A username created using the CLI may not be used to log into the device.

Condition:
The username has to be 17 characters or longer.

Workaround:
Create usernames that are no more than 16 characters in length.

LRS-1165 (Moderate) Deleting a more precise route that masks a direct attach route may direct traffic to wrong route

Symptom:
Traffic gets directed to the wrong route.

Conditions:
A direct attach route (interface route) exists, and then a more specific route was added and then deleted.

Workaround:
Deleting and re-adding the direct attach route or using "clear ip route *" may correct the issue.

Example:
If the configuration contains a default route and also a static route which has an interface as its destination, such as:
ip route 0.0.0.0/0 10.0.0.1
ip route 10.201.0.0/16 ix0
Then a second route that is a more specific route within the first route is added:
ip route 10.201.0.1/32 ix1
And then deleted:
no ip route 10.201.0.1/32 ix1
Then traffic to 10.201.0.1 will be directed to the default route instead of correctly being directed to interface ix0.

LRS-1206 (Moderate) Subinterfaces don't come up until physical interface is upped

Symptoms:
Subinterfaces do not pass traffic until the physical interface associated with the subinterfaces has been upped.

Conditions:
Only the subinterfaces have to be configured with an IP address.

Workaround:
Bring up the physical interface associated with the subinterface using the no shutdown command.

LRS-1248 (Moderate) Routes requiring recursive route resolution only work intermittently

Symptoms:
Routes requiring recursive route resolution may or may not be installed properly into the forwarding information base. The show ip route command displays the installed routes. Even if such a route is installed correctly, changing interface IP addresses or unrelated routes may cause it to not be installed properly.

Conditions:
One or more routes requiring recursive route resolution are configured.

Workaround:
Manually resolve routes that would otherwise need recursive route resolution and manually configure the resolved route.

Additional Information:
An example problematic configuration:

interface ix0
ip address 10.1.2.3/16

ip route 10.2.0.0/16 10.1.0.1
ip route 10.3.0.0/16 10.2.0.1

Notice that the gateway for the 10.3.0.0 route can only be reached by recursively resolving the IP route 10.2.0.0/16 route. This will only function correctly if the 10.2.0.0/16 route is installed before 10.3.0.0/16, which is not guaranteed by the system.

Recursive resolution with respect to direct attach routes is sufficient. The following will function correctly as direct attach routes are installed prior to gateway routes, regardless of configuration order in show run:

ip route 10.2.0.0/16 ix0
ip route 10.3.0.0/16 10.2.0.1

LRS-1251 (Moderate) help systems erroneously indicates that encapsulation is a valid option for primary interfaces

Symptoms: Online help system offers "encapsulation" as a valid option for primary interfaces (for example, ix0) when they are only valid on sub-interfaces, such as ix0.1

Conditions:
View online help for any primary interface.

Workaround:
None.

Additional Information:
When attempting configure encapsulation for a primary interface, the system responds with an error that says you can only configure encapsulation on a subinterface.

LRS-1252 (Moderate) Help system erroneously offers flowcontrol as a valid option on sub-interfaces

Symptoms:
Online help system offers "flowcontrol" as a valid option for subinterfaces (for example, ix0.1) when they are only valid on primary interfaces, such as ix0.

Conditions:
View online help for any subinterface.

Workaround:
None.

LRS-1262 (Moderate) syslog: Cannot use "level debug" under "logging filter <filtername>" configuration

Symptoms:
Entering the configuration command "level debug" or "no level debug" subcommands under the "logging filter <filtername>" command displays the error message - ERROR "debug" not recognized.

Conditions:
None

Workaround:
The workaround is to use the numeric value of the level "debug", which is 7, instead of the word "debug".

For example
host(config)# logging filter filter1
host(config-log-filter)# level 7

LRS-1270 (Moderate) Show int shows incorrect subinterface MTU when encapsulation not set on subinterface

Symptom:
Show interface shows 1500 for a subinterface MTU, even though the MTU is configured to a different number on that subinterface.

Condition:
A subinterface has been created, but no "encapsulation" configuration has been set on the subinterface.

Workaround:
The subinterface interface cannot go to the up state until "encapsulation" is set, so the incorrect MTU will not affect any network traffic. Adding "encapsulation" to the subinterface fixes the problem, and the correct MTU is set.

LRS-1290 (Moderate) snmpd: error finding row index in _ifXTable_container _row_restore

Symptom:
The following message appears in /var/log/messages at boot time:

snmpd[<process ID>]: error finding row index in _ifXTable_container_row_restore

Conditions:
Unknown

Workaround:
None known

LRS-1337 (Moderate) All kernel messages are logged with level "notice"

Symptoms:
All kernel messages are being logged only at log level "notice."

Conditions:
Any kernal messages logged.

Workaround:
None.

LRS-1338 (Moderate) Sent multicast/broadcast packets are not being counted for VLAN sub interfaces

Symptom:
The sent multicast/broadcast statistic is not incrementing for VLAN subinterfaces.

Condition:
System configured for VLAN subinterfaces

Workaround:
None.

LRS-1359 (Moderate) SNMP: Internal error: Path not found message when enabling SNMP on an IP address

Symptom:
Configuring snmp-server to listen on an IP address displays the error message: "ERROR: Unable to enable snmp agent on udpv4 <ipaddr>Internal error: Path not found" in the CLI.

Condition:
System configured to use SNMP.
Two lros-shell sessions need to be active, and one deletes the snmp-server configuration using "no snmp-server", while the second one is in the "snmp-server" configuration mode.

Workaround:
None.

Additional Information:
Modifying the same config at the same time from two different lros-shell sessions is discouraged.

LRS-1371 (Moderate) SNMP server listens on default UDP port on all IPv4-enabled interfaces when not configured

Symptoms:
The system is listening on the default SNMP UDP port (161) on all IPv4-enabled interfaces, even though snmp-server is not configured.

Conditions:
No snmp-server configuration is present in the running-config.

Workaround:
None.

Additional information:
Although the SNMP server is listening on the ports, the default snmp-server configuration does not have any community strings configured, so no SNMP queries will be accepted by the SNMP server. All incoming SNMP packets will be dropped by the SNMP server.

LRS-1390 (Moderate) Routes can be removed that are not in the running configuration

Symptom:
Users can remove a route that is not in the current running configuration if they specify the interface that the gateway of the route is configured on. For example the route is configured with "ip route 192.168.1.0/24 192.168.2.1" but can be removed with "no ip route 192.168.1.0/24 ix0".

Condition:
System configured for static routes with IP address gateways.

Workaround:
None. Take care when removing routes with the gateway.

LRS-1392 (Moderate) Emulex: Show interface shows Auto-speed instead of actual negotiated speed

Symptom:
Show interface output shows "Auto-speed" instead of the negotiated speed for oce interfaces.

Condition:
System running on a host with Emulex NICs that use the ocenet driver.

Workaround:
None

Additional Information:
You can verify the speed of the link by checking the interface of the device the system is connected to. For example, by using the "show interface" command on Cisco or Arista switches.

LRS-1407 (Moderate) Show interface does not show configured IP addresses

Symptom:
The show interface command does not show IP addresses that are configured on the interface.

Conditions:
This may occur after configuring two IP addresses that are in the same IP subnet on the interface.

Workaround:
Remove IP addresses from the interface such that no two IP addresses configured on the interface are in the same IP subnet. Save the configuration and reload the system.

LRS-1432 (Moderate) Real Server URL hash does not recompute upon server address change

Symptoms:
URLs may get hashed inconsistently to different real servers.

Conditions:
If the IP address or port of a real server is changed, the load balancer does not immediately update the hashing algorithm with that change. So the particular server that any given URL hashes to may differ between load balancer processes, if any process has terminated and restarted since the IP address or port change. Also, a reboot of the system after the IP address or port change may also cause the URLs to hash to different servers.

Workaround:
None.

LRS-1449 (Moderate) OCE interfaces support a maximum of 64 VLANs

Symptom:
Configuring more than 64 interfaces with VLANs results in additional interfaces that do not pass traffic, and the following logs are seen in the /var/log/messages file"
'kernel: ocenet_register_vlan: port=0, too many vlans=65326'

Condition:
System running on a server with Emulex interface cards.

Workaround:
None. The current limit on the number of supported VLAN interfaces is 64.

LRS-1450 (Moderate) HTTP CONNECT statistics should be displayed as part of "show real-server <server> statistics"

Symptoms:
HTTP CONNECT statistics do not show in "show real-server <servername> statistics [detailed]" output.

Conditions
None.

Workaround:
There is no workaround to view real server-specific statistics.
In version 1.6.x, use "show load-balancer statistics detailed" and in version 2.x, use "show proxy statistics detailed" to see all the aggregated HTTP CONNECT statistics.

LRS-1472 (Moderate) VLAN interfaces have intermittent connectivity after initial configuration

Symptom:
Intermittent network connectivity after configuring VLAN interfaces.

Condition:
System configured for VLAN interfaces.

Workaround:
None. The condition appears to resolve itself after approximately 45 seconds.

LRS-1481 (Moderate) BCE interfaces take 3-4 seconds to come up after down/up

Symptom
BCE interfaces take 3-4 seconds to become active after enabling them (no shutdown).

Condition
None.

Workaround
None. Interfaces will come up after 3-4 seconds.

LRS-1482 (Moderate) Error setting /app/lb_http/rserver/rs1/hmonitor/h1/status: Internal error: Path not found

Symptom
The message "Error setting /app/lb_http/rserver/rs1/hmonitor/h1/status: Internal error: Path not found" is seen in the logs if a health monitor probe receives connection refused.

Condition
System configured for health monitor, and the real server refuses the connection.

Workaround
None.

LRS-1523 (Moderate) virtual-ip HTTP response 5xx statistics inconsistent with total number of responses sent

Symptoms:
The virtual IP statistics for 5XX errors and total responses may be inconsistent with one another. These statistics can be viewed with the "show virtual-ip <name> statistics" command.

Conditions:
Certain errors in HTTP responses received from a real server may cause the counters to be updated incorrectly.

Workaround:
None.

LRS-1558 (Moderate) Cannot use CTRL+Z to exit config mode while entering a pem-format certificate

Symptom:
Pressing the CTRL+Z keys to exit configuration mode while entering a PEM format certificate or key does not exit the configuration mode.

Condition:
System with user configuring a PEM format certificate or key.

Workaround:
Type quit on a line by itself to complete the PEM format command, and then use the CTRL-Z command.

LRS-1568 (Moderate) Client for SSL termination cannot validate certificate chain more than 9 deep

Symptom:
Clients fail to verify certificate chains that are more than nine deep.

Condition:
System configured for SSL termination with certificate chains that are more than nine deep.

Workaround:
None.

Additional Information:
The curl command shows the following error message: SSL certificate verify result: unable to get local issuer certificate (20).

LRS-1603 (Moderate) "Allow to" rules cannot be enabled on multiple ports for a single IP address.

Symptoms:
System fails to preserve SSH, REST server, or SNMP server "allow to" rules when the same IP address is used with different port numbers. Instead of adding a new "allow to" rule for each "IP address / port number" combination, the system overwrites the existing "allow to" rule for the given IP address.

Conditions:
None

Workaround
None

LRS-1647 (Moderate) Strings are not quoted in the configuration prompt for sub-modes

Symptom:
Strings that were quoted in the configuration are not quoted in the sub-mode configuration prompt.

Condition:
Configuration of commands that use quoted strings for their sub-configuration mode commands.

Workaround:
None.

Example Output:

host-1-11(config)# health-monitor "Test H M"
host-1-11(config-hmonitor:Test H M)#

LRS-1658 (Moderate) System may send duplicate TCP ACK packets in some circumstances

Symptom:
The system may send a duplicate and unnecessary TCP ACK packet in some rare cases.

Conditions:
Triggering conditions are unknown, but appear to be rare.

Workaround:
None known.

LRS-1698 (Moderate) SSL Profile shows RC4-MD5 twice

Symptom:
After entering an OpenSSL cipher string of RC4-MD5, it is shown twice is the ordered cipher list section of the show ssl profile <profile> command.

Condition:
System configured for SSL profiles that use the OpenSSL cipher string RC4-MD5.

Workaround:
None.

LRS-1699 (Moderate) SSL Version list is shown as <none> default in the output of show ssl profile

Symptom:
The output of the show ssl profile <profile> command shows the SSL version list as none. No option currently exists to modify the supported SSL protocol versions allowed.

Condition:
An SSL profile has been configured.

Workaround:
None needed. The information shown in the show command is extraneous.

LRS-1702 (Moderate) Health monitor cannot be removed from a real-server if it is using a base that includes the health monitor

Symptom:
There is no configuration command to disable health monitoring from a real server if the real server has inherited its health monitor configuration from a real server base.

Condition:
System configured with health monitoring that is attached to real servers via bases.

Workaround:
The base must be removed from the real server to remove the health monitor. If this is done, all configuration from the base must be added to the real server (except the health monitor) to ensure it has the same configuration as the other real servers.

LRS-1706 (Moderate) Ejecting root filesystem may panic the kernel

Symptom:
Kernel panics after forcibly removing the media containing the root filesystem. For example, removing the USB drive, ejecting a CD ROM, or unmounting virtual media.

Conditions:
Removing the media containing the root filesystem.

Work Around:
Shut down the system first, and then remove the media containing the root filesystem.

LRS-1755 (Moderate) VLAN interface shows zeros for MAC address

Symptom:
Show interface shows all zeros for the MAC address for a VLAN subinterface.

Conditions:
Subinterface configured on a physical interface and no "encapsulation" is set on the subinterface.

Workaround:
The MAC address may be found on the physical interface associated with the subinterface. The MAC address will also be correct after the encapsulation is set on the subinterface.

LRS-1760 (Moderate) Kernel waits indefinitely to flush data when the remote connection advertises a 0 TCP window.

Symptoms:
Connections that should be closed show in the ESTABLISHED state and continuously attempt to flush with no timeout, even if a response timeout is set.

Conditions:
The remote side of a TCP connection indefinitely advertises a 0 TCP window, and there is data pending in the kernel.

Workaround:
None.

LRS-1767 (Moderate) Interface MTU change may result in traffic sent to wrong interface briefly

Symptom:
For a very short window of time, IP traffic flowing through or originating from the system may forward out the wrong interface.

Conditions:
The MTU configuration for any interface on the system is changed by the administrator.

Workaround:
None.

Additional Information:
When the MTU for an interface changes, all static IP routes are cleared and re-added automatically. Because forwarding of IP traffic is not disabled during this time, the order in which the routes are cleared and re-added may allow IP packets to forward to a less specific route (such as the default route) than they should.

LRS-1774 (Moderate) Health monitor of HTTPS real servers behaves poorly for more than 1,000 servers

Symptom:
Real servers are are intermittently marked down and back up when there are no actual problems with the real servers.

Condition:
Health monitoring is configured to monitor a large number of HTTPS real servers, or the real servers are using large key sizes.

Workaround:
Increase the health monitor interval and timeout values. For example, 20-second interval and 3-second timeout.

Additional Information:
With 1,024 bit RSA keys, the health monitor has been successfully tested to 1,000 servers using the default interval and timeout. With larger key sizes, the health monitor will not be able to monitor as many servers without reducing interval and timeout values.

LRS-1790 (Moderate) Performance degradation with repeated add/remove of HTTP strip header configuration

Symptom:
HTTP connections per second performance drops by up to about 5%.

Conditions:
Repeated addition and removal of multiple different "strip-request-header" header names.

Workaround:
Removal and re-creation of virtual server or periodic reload of device.

Additional information:
This issue only occurs when using many different header names. Addition and removal of a small set of the same header names does not cause performance degredation.

LRS-1838 (Moderate) CARP: unexpected errors in client traffic after forcing CARP failover

Symptoms:
HTTP clients may see errors for a short time after a CARP failover.

Conditions:
Two or more systems configured together in a CARP group, and then failure of the master of that CARP group, so that a backup system takes over.

Workaround:
None known.

Additional information:
Some client errors due to connection loss are expected during a CARP failover, but the errors may continue for a short time period longer than expected.

LRS-1857 (Moderate) Warning messages from commands may be repetitive

Symptom:
In response to a shell command, many repetitive warning messages appear.

Conditions:
The user executes a shell command, which produces a large quantity of warning messages.

Workaround:
None.

LRS-1870 (Moderate) CARP: group should be automatically removed when all carp commands are removed from interface subconfig

Symptoms:
The "show carp" command displays CARP groups attached to an interface even after removing all CARP configuration for that group.

Conditions:
A CARP group was configured on an interface and then each individual CARP configuration was removed from that group.

Workaround:
Manually remove CARP group via: "no carp <group>".

LRS-1891 (Moderate) CLI: parser needs strict text matching when using 'no' to remove config

Symptom:
Some CLI commands ignore arguments when using the "no" form of the command to remove configuration from the device.

Conditions:
Administrator issues a command in the form of "no <command> <keyword> <ignored text>"

Workaround:
None

LRS-1953 (Moderate) login_getclass message on boot

Symptoms:
The following message appears at system boot time:
init: login_getclass: unknown class 'daemon'

Conditions:
No additional conditions required.

Workaround:
None known.

Additional information:
This message appears to have no effect on the correct operation of the system.

LRS-1954 (Moderate) Device asks to save configuration when no configuration changes have been made

Symptom:
After booting a system and making no configuration changes, then doing a reload, install, or upgrade, the system may say that configuration changes have been made and ask if you want to save those changes.

Conditions:
None required,

Workaround:
None known.

Additional information:
In the LROS shell, the current differences between the running-config and startup-config can be viewed with 'show running-config diff'. This shows those items that the system has determined are changes and are in need of saving.

LRS-1961 (Moderate) In "netstat -r" output, the "Netif" column is too narrow

Symptoms:
The "netstat -r" command (used to show network routes) uses fixed-width columns in its output. The column showing the network interface name associated with each route is limited to six characters. If a route goes out a network interface with a name longer than six characters, the network name will be truncated, and only the first six characters will show up.

Conditions:
This is only a problem when a system has network interface names longer than six characters.

Workaround:
Use network interface names that are six characters or shorter, or ensure that the first six characters of the interface names are unique.

Additional Information:
No harm is done beyond confusion in the output of 'netstat -r'. Specifically, routing still works correctly with long interface names.

LRS-1979 (Moderate) System may take 60 seconds to gain a performance benefit from reducing real-server max-connections

Symptoms:
A change to real server max-connections may take more than 60 seconds to show meaningful change in overall system performance.

Conditions:
A change is made to either the real server or real server base max-connections parameter.

Workaround:
None known.

Additional information:
The change to the max-connections parameter takes effect immediately. However, when reducing max-connections, current connections that are open will not be forcibly closed. No new connections will be opened to a real server until its current connection count falls below the max-connections parameter setting. This may be the cause of this issue.

LRS-1987 (Moderate) Performance drop in 1.3.0 when removing and reapplying system configuration using a configuration file

Symptoms:
~1K performance drop seen in connections/sec when adding/removing system configuration.

Conditions:
Add/remove virtual-ip and virtual-server configuration using a configuration file from bash, for example, lros_shell --config <somefile>.

Workaround:
Reboot after non-trivial configuration operations, such as bulk configuration add/remove.

LRS-2001 (Moderate) Error message in log: mld6_input: src :: is not link-local (grp=::)

Symptom:
Error messages similar to the following may be seen in the system logs:

mld6_input: src :: is not link-local (grp=::)

Conditions:
Unknown.

Workaround:
None known.

LRS-2101 (Moderate) IP Fragment stats are maintained inconsistently

Symptoms:
The IP fragment statistics (available via sysctl) are not perfectly reliable. The reported numbers are only approximate.

Conditions:
Using sysctl to view IP fragment statistics.

Workaround:
None.

Additional Information:
IP fragments are handled correctly, but the statistics about them are counted incorrectly.

LRS-2106 (Moderate) Setting "service TCP" on a virtual server results in: ERROR: ServiceTypes do not match

Symptoms:
Setting "service tcp" on a virtual server fails with error "ERROR: ServiceTypes do not match".

Conditions:
The virtual server has a virtual IP or real server attached to it that is of type "service http" and is online ("admin-status online").

Workaround:
Set "admin-status offline" on all attached real servers and virtual IPs before setting "service tcp" on the virtual server.

LRS-2111 (Moderate) When configuration objects are entered into the configuration, they should show their default to be of type http.

Symptom:
After entering a real-server, virtual-server, or virtual-ip command the service type is not shown in the configuration.

Condition:
System configured with real-server, virtual-server, or virtual-ip commands with no options.

Workaround:
Manually use the "service http" command, which is the default service type.

LRS-2309 (Moderate) Plugging in a USB DVD drive may cause a kernel panic

Symptom:
Kernel panic after inserting a USB DVD drive into a running system.

Condition:
Inserting a USB DVD drive into a running system.

Workaround:
None. Remove USB DVD drive and reboot the system.

LRS-2322 (Moderate) Request idle timeout and response timeout only take effect after the currently in flight request completes or the prior timer expires.

Symptoms:
Certain requests and responses fail to honor the request and response idle timeouts.

Conditions:
The timeouts were modified after the request was issued/response initiated.

Workaround:
None.

LRS-2327 (Moderate) Real servers are not shown in virtual server output even if attached with a real server group

Symptom:
Real servers are shown as "<none>" in the output of "show virtual-server <vserver>" if attached using real server group.

Condition:
System configured for virtual servers that have real server groups attached to them.

Workaround:
Use the "show real-server group" command to see which real servers are attached.

LRS-2348 (Moderate) Virtual IP addresses with same IP/port are allowed to be admin status online at the same time.

Symptom:
Configuring a second virtual IP with the same IP/port as an existing virtual IP is allowed by the system.

Condition:
None.

Workaround:
None. Ensure that you are not configuring duplicate virtual IP entries.

LRS-2374 (Moderate) Service tcp virtual-servers show url-hash as a valid lb-algorithm

Symptom:
The "url-hash" is shown as a valid lb-algorithm option when a virtual server is configured as service TCP.

Condition:
System configured with TCP virtual servers.

Workaround:
None. An error message displays if you configure "url-hash" as the lb-algorithm for a TCP virtual server.

LRS-2375 (Moderate) Real server max-connections is not working

Symptom:
Connections to the real server exceed the number configured by max-connections for the real server.

Condition:
System configured with TCP real servers.

Workaround:
None. This is due to the number of load balancing processes that the system is running. The number of connections will not exceed the number of configured max-connections plus the number of load-balancing (version 1.6.x) or proxy (version 2.x) processes.

LRS-2377 (Moderate) Actual open connections may be higher than configured real server max-connections

Symptoms:
The number of open connections to a real server may be higher than the configured value for max-connections on the real server.

Conditions:
The system is configured for load balancing, max-connections is configured on a real server, and max-connections is not an even multiple of the value for 'load-balancer processes' (version 1.6.x) or 'proxy processes' (version 2.x).

Workaround:
Set max-connections to an even multiple of the value for load-balancer processes (version 1.6.x) or proxy processes (version 2.x).

Additional information:
Each load balancer data path process in the system is able to open a number of connections to each real server. The number that each process can open is the value of max-connections for that real server divided by the number of load balancer or proxy processes, rounded up to the nearest integer. This can result in the load balancer opening a few additional connections to each real server.

LRS-2384 (Moderate) IP Filter: show ip filter-list brief does not show correct number for Compiled Rules

Symptoms:
Incorrect value displayed for Compiled Rules.

Conditions:
Compiled Rules is 2 greater than the true number of rules in an ip filter-list.

Workaround:
Automated tools will need to subtract 2 from the Compiled Rules value.

LRS-2521 (Moderate) ERROR: Unable to create data directory /home/linerate/data: Error 256: mkdir: /home/linerate/data: No such file or directory

Symptom:
The error message "LROS: ERROR: Unable to create data directory /home/linerate/data: Error 256: mkdir: /home/linerate/data: No such file or directory" is seen in /var/log/messages after boot up.

Condition:
None.

Workaround:
None.

LRS-2652 (Moderate) client-ip-header does not accept all characters, even if they are quoted properly.

Symptoms:
The client-ip-header command (service HTTP setting for a virtual server) does not accept all characters, even when surrounded by double quotes.

Conditions:
Enter characters that are not legal for a Word parameter type, surrounded by quotes, in the client-ip-header command.

Workaround:
Use only legal characters for a Word parameter type (upper and lower case letters, digits, dashes, and periods).

LRS-2663 (Moderate) Process manager needs a mechanism to attempt to restart a process (that died unexpectedly) more than once if necessary

Symptoms:
The total number of load balancer data path processes (lb_http processes) that is running on the system is less than the number configured. Also, messages similar to the following will be present in the system logs:
LROS: ProcManager: unable to restart lb_http

Conditions:
System is configured for load balancing and one or more load balancer data path processes have died for any reason.

Workaround:
None known.

LRS-2669 (Moderate) Log message: WARNING: / was not properly dismounted

Symptoms:
Messages similar to the following may be seen in the system logs at boot time:
Trying to mount root from ufs:ufs/LROS5210b4a3e7349f3rootfs
WARNING: / was not properly dismounted
WARNING: / was not properly dismounted

Conditions:
System has been rebooted. This may be specific to a reboot after the upgrade command.

Workaround:
None known.

LRS-2676 (Moderate) REST: PUT to create multiple virtual-ip interfaces doesn't return error message

Symptom:
Creation of configuration fails but no error is returned from the REST API.

Conditions:
POST or PUT a JSON object containing multiple nodes.

Workaround:
Multi-node JSON objects are not currently supported. The JSON must be subdivided to create or update a single node at a time.

LRS-2744 (Moderate) The GUI does not support browser-based navigation (navigating through current session using the browser's Back and Forward buttons)

Symptom:
System does not allow navigation within the application using standard browser Back and Forward buttons. Clicking the browser's Back or Forward button may exit the GUI.

Condition:
Clicking the browser's Back or Forward button.

Workaround:
Use the filter strip to control navigation. Deleting a filter item in the filter strip will restore the previous view, similar to clicking a browser Back button.

LRS-2765 (Moderate) libs/uiproto/client.cc(45): Throw in function void lrs::uiproto::UIProtoClient::connect()

Symptom:
Exception when applying an SSL profile base with a very long name to an SSL profile.

Condition:
System configured with SSL profiles that use SSL profile bases with very long names (greater than 256 characters).

Workaround:
Use a name for the SSL profile base that is less than 256 characters.

LRS-2798 (Moderate) REST: controller disconnected unexpectedly when sending 99 MB JSON object

Symptom:
Controller failure in REST API during PUT or POST operation.

Conditions:
Send a valid JSON object containing a very large data field in the object.

Workaround:
Client is required limit size of JSON objects. Reasonable object size is <10 MB.

LRS-2800 (Moderate) REST: server returns 500 with no response content

Symptom:
HTTP response code 500 in REST request when attempting to create a virtual IP.

Conditions:
Badly formatted JSON object send in POST request.

Workaround:
None.

Additional Information:
REST is expected to return a meaningful error code and description in response to a malformed request.

LRS-2819 (Moderate) LROS: netstat command fails occasionally with memory allocation error

Symptom:
Using netstat command from bash prompt returns the following error: "netstat: sysctl: net.inet.tcp.pcblist: Cannot allocate memory"

Conditions:
System is under moderate to heavy load, resulting in low free memory available

Workaround:
This is a transient issue while the system is under load and does not affect data traffic. Retry the command.

LRS-2840 (Moderate) CLI says token "is ambiguous" on syntax errors

Symptom:
CLI says some errors are due to ambiguous input when they are actually due to syntax errors.

Conditions:
A CLI command is entered that contains a variable (like <WORD> or <INTEGER> ). The command as entered has a syntax error: the incorrect type of variable is entered (for instance, a string is entered when only a word is allowed, or a word is entered when only an integer is allowed).

Workaround:
Enter a command with the proper syntax. Use '?' to for help, to know what types of variables are allowed in the command you are entering.

LRS-2855 (Moderate) When removing an SSL profile, system should warn if it is in use by configuration objects

Symptom:
No warning message when removing an SSL profile that is in use.

Condition:
System configured with objects that have SSL profiles attached to them.

Workaround:
None. Because of late binding, this is allowed.

LRS-2900 (Moderate) Error when running show command on SSL profile with long name

Symptoms:
Running 'show ssl profile <ssl-profile-name>' on an SSL profile with a name longer than 245 characters results in an error message similar to the one below:
...
Ordered cipher list
ERROR: Unable to retrieve data from controller:
Path: <ssl-profile-name>/cipherNameList

Also, messages similar to the following will be present in the system logs:
Caught exception getting cipherNameList

Conditions:
System is configured with an SSL profile with a name longer than 245 characters.

Workaround:
Configure SSL profile with names that are 245 characters or less.

LRS-2920 (Moderate) "show debug sysdb value <name>" with bad path exits shell

Symptoms:
Shell unexpectedly exits after executing "show debug sysdb value <name>", with the following message:

host-name# show debug sysdb value "/app"
ERROR: Exiting on exception:
libs/uiproto/lrs/uiproto/client.tcc(122): Throw in function std::string lrs::uiproto::UIProtoClient::getAsString(const std::string&, bool, double)
Dynamic exception type: boost::exception_detail::clone_impl<lrs::uiproto::UnhandledType>
std::exception::what: std::exception
[lrs::uiproto::tagUIPath*] = /app
[lrs::uiproto::tagObjType*] = 12
Connection to host-name closed.

Conditions:
Execute "show debug sysdb value" with an non-existant sysdb path

Workaround:
Note: Consider carefully before using this workaround, because this command can have a serious impact on your system. For information about the command discussed below, see https://docs.lineratesystems.com/200Release_2.0/200CLI_Reference_Guide/Exec_Commands/Show_Commands/Show_Debug_Commands.

Log back into the shell, and only use existing paths with "show debug sysdb value <name>".

Additional Information:
Some paths will not cause the shell to exit, but instead display the following error:

ERROR: Could not retrieve '/path/that/does/not/exist;
'Get' request failed:
ResponseCode: 101 (Internal error: Path not found)

LRS-2969 (Moderate) Tab completion doesn't print help when 1 keyword and CR are candidates

Symptom:
Sometimes triggering Tab completion when the completion is ambiguous doesn't print help.

Conditions:
Type a partial command at the prompt without pressing Enter. Press the Tab key twice to trigger printing help on an ambiguous tab completion. Help is not printed. This must happen at a level where one of the possible command completions is just pressing Enter. The command is a valid command, but it could be extended into a longer command as well, for example by adding "detail".

Workaround:
Press "?" to get the help manually.

LRS-2994 (Moderate) Removing a route from the configuration does not remove the route from the system

Symptoms:
Removing a route from the config does not remove it from the system. The route is removed from the configuration as indicated by "show run".

Conditions:
1) Add a route to a network without using zeros (0) in the host portion of the network. For example 3.4.5.6/24 instead of 3.4.5.0/24
2) Remove the route added in #1 from the config. An error message similar to the one described in LRS-2993 is generated, but the route to the 3.4.5.0/24 net is not removed from the system. This can be verified either using netstat or by pinging a host on the 3.4.5.0/24 network.

Workaround:
While adding a route to a network, make sure that the host portion of the network is all zeros.

LRS-3023 (Moderate) SNMP error message: Could not restart snmp agent

Symptom:
Error messages seen in logs similar to the following:
LROS: Error activating snmp changes. Will retry on the next config change
LROS: Details: controller/modules/snmp/snmp_writer.cc(27): Throw in function virtual void lrs::ctrl::NetSnmpWriter::activateChanges()
LROS: Dynamic exception type: boost::exception_detail::clone_impl<ex::InternalErr>
LROS: std::exception::what: std::exception
LROS: [ex::tagUserMsg*] = Could not restart snmp agent

Conditions:
System is configured to enable SNMP server.

Workaround:
None known.

LRS-3054 (Moderate) GUI filter does not support "==" as a valid equality operator.

Symptom:
User receives an error when typing an equals operator as '=='.

Condition:
Typing '==' for an equals operator in filters (for example, name==vip1 will generate an error because the system only supports the '=' symbol for equality filters).

Workaround:
Use a single '=' character for equality filters (name=vip1).

LRS-3137 (Moderate) show backup list detail shows all backups as type "partial"

Symptoms:
Output from the "show backup list detail" command lists all backups as type "partial." This makes it impossible to know which backup is partial and which backup is full.
linerate01# show backup list detail
Available backups:
Backup Name Type Version
-----------------------------------------------
backup-date-partial.tar.bz2 partial N/A
backup-date-full.tar.bz2 partial N/A

Conditions:
System with full backups.

Workaround:
Use bash shell to compare backup file sizes. A full backup will be several MB in size. A partial will typically be less than 1 MB.

linerate01# bash
[admin@linerate01 ~]$ ls -l backups/
total 41840
rw-rr- 1 root lros 42777718 Apr 20 04:30 backup-date-full.tar.bz2
rw-rr- 1 root lros 14766 Apr 20 04:27 backup-date-partial.tar.bz2
[admin@linerate01 ~]$ du -hs backups/*
41M backups/backup-date-full.tar.bz2
16K backups/backup-date-partial.tar.bz2

LRS-3147 (Moderate) Virtual IP admin status inconsistent if exception

Symptoms:
Virtual IP may be reported as being online when it is not.

Conditions:
If an error is reported when setting a virtual IP's admin status to online, the virtual IP may report its admin status is online when it is not.

Workaround:
Set the admin status to offline, then to online again. The issue will persist until the admin status is set to online without errors.

LRS-3153 (Moderate) Average values displayed as part of the "show real-server <name> statistics" are not entirely accurate

Symptom:
The average values displayed in "show real-server <name> statistics" are not entirely accurate.

Conditions:
None.

Workaround:
None.

Additional Information:
The averages are off by a small factor due to a minor computational error. This issue is applicable to the average values reported in virtual IP and virtual server statistics as well. If the system has X number of load-balancer (version 1.6.x) or proxy (version 2.x) processes, the correct average value can be obtained by multiplying the reported value with ((X+1)/X).

LRS-3174 (Moderate) Need config migration when downgrading to earlier versions

Symptom:
Unable to connect to system via SSH after booting to a down-level version of LROS.

Conditions:
Use upgrade command to install down-level version. The upgrade command copies the current running-config to the startup-config for the next version to be booted after installation completes.

Workaround:
Connect via console and manually fix config after system reboot. Downlevel versions may not support all features of the current version. Due to current CLI behavior ("delayed config"), a top-level config command will not be accepted and subconfig commands will be applied as part of the previous top-level config. For instance, the current introduced rest-server. Using upgrade to boot back to the previous version will result in rest-server subconfig commands, such as "allow from" and "allow to", being applied to the SSH configuration.

LRS-3203 (Moderate) GUI does not quickly identify misconfigurations and incomplete configurations.

Symptoms:
Current GUI filtering capabilities don't identify common system misconfigurations or incomplete configurations.

Conditions:

  • Config contains objects with duplicate IP addresses (that is, virtual IP or real server objects).
  • Config contains a real server attached to multiple virtual servers.
  • Config contains virtual servers with no default host (even though the "host names" list is not empty).
  • Config contains objects that inherit from non-existent bases (that is, virtual IP, real server, or SSL profile objects).
  • Config contains "orphaned" objects (that is, objects that aren't associated with any other object).

Workaround:
None.

LRS-3204 (Moderate) Kernel Panic during reboot after install

Symptoms:
Errors reported on console during reboot after install:

vm_fault: pager read error, pid 1 (init)
...
exit1() at exit1+0xa0
sys_exit() at sys_exit+0xe
syscall() at syscall_0x254
Xfast_syscall() at Xfast_syscall+0xab
— syscall (1, FreeBSD ELF64, sys_exit), rip = 0x67719c, rsp = 0x7fffffffe078, rbp = 0x6c6 —

Conditions:
Install media was removed before system has fully shut down.

Workaround:
Leave install media in until the system has completely shutdown, and the BIOS boot messages are displayed.

Additional Info:
These errors can be ignored if the install media was in until the "Reboot and remove CD now" message was displayed.

LRS-3249 (Moderate) System fails to process filter conditions where the target value includes a character from operators list (=, ~, !, <, >).

Symptoms:
System fails to add a new filter condition to the filter chain. Press Enter, but nothing happens.

Conditions:
Create a filter condition where the target value includes one of the operator characters (for example, VIP name = xxabc==).

Workaround:
None.

LRS-3263 (Moderate) Ethernet link goes down briefly after executing 'ip routing' or 'no ip routing'

Symptom:
Network traffic is disrupted for up to a few seconds, potentially causing errors seen by clients and servers connected through the system.

Conditions:
Executing the commands "ip routing" or "no ip routing".

Workaround:
Configure the setting for "ip routing" in startup-config and do not change the value while traffic is being processed.

Additional information:
The 'ip routing' and 'no ip routing' configuration commands result in all Ethernet interfaces transitioning to link down state briefly. Some interfaces may take a longer time to re-establish link than others.

LRS-3304 (Moderate) The system displays default values for HTTP-only attributes in a TCP object row or TCP-only attributes in an HTTP object row.

Symptoms:
Table pane displays default values for HTTP-only attributes in TCP object rows (the displayed values don't apply to the TCP object). Table pane displays default values for TCP-only attributes in HTTP object rows (the displayed values don't apply to the HTTP object).

Conditions:
Example:

  • System config contains a virtual IP whose Service Type is TCP.
  • Note that this TCP virtual IP's row in the Virtual IP table contains non-blank values for the HTTP-only attributes Keepalive Timeout, Max Header Size and Max In Flight.

Workaround:
None.

LRS-3315 (Moderate) Node.js exception when user does not exist in /var/log/messages

Symptom:
Node.js exception in the log files for a user that does not exist.

Apr 17 13:44:49 host-44-data LROS: Error: User does not exist
Apr 17 13:44:49 host-44-data LROS: at findUserById (/usr/linerate/node/node_js/http_serv.js:38:8)
Apr 17 13:44:49 host-44-data LROS: at /usr/linerate/node/node_js/http_serv.js:52:3
Apr 17 13:44:49 host-44-data LROS: at pass (/usr/linerate/node/node_modules/passport/lib/passport/index.js:217:7)
Apr 17 13:44:49 host-44-data LROS: at Passport.deserializeUser (/usr/linerate/node/node_modules/passport/lib/passport/index.js:221:4)
Apr 17 13:44:49 host-44-data LROS: at SessionStrategy.authenticate (/usr/linerate/node/node_modules/passport/lib/passport/strategies/session.js:40:28)
Apr 17 13:44:49 host-44-data LROS: at Passport.authenticate (/usr/linerate/node/node_modules/passport/lib/passport/middleware/authenticate.js:114:14)
Apr 17 13:44:49 host-44-data LROS: at Object.handle (native)
Apr 17 13:44:49 host-44-data LROS: at next (/usr/linerate/node/node_modules/connect/lib/http.js:203:15)
Apr 17 13:44:49 host-44-data LROS: at Passport.initialize (/usr/linerate/node/node_modules/passport/lib/passport/middleware/initialize.js:69:5)
Apr 17 13:44:49 host-44-data LROS: at Object.handle (native)

Condition:
User logs in to GUI with a username that is not configured on the system.

Workaround:
None.

LRS-3335 (Moderate) Question help gives empty list of options

Symptom:
Occasionally, using the question mark in the command line interface will result in an empty list of help options.

Conditions:
The only place in the command line interface where this is known to occur currently is in virtual-server configuration mode when attaching a real server group and when typing a question mark after the "weight" keyword, like below:

host-lros(config-vserver:vs1)# attach real-server group mygroup-vs1 weight ?

Available options are:
host-lros(config-vserver:vs1)#

Workaround:
None known.

LRS-3366 (Moderate) ssh and rest-server 'allow from' command does not allow specifying a single IPv6 address without a prefix

Symptoms:
The 'allow from' option associated with 'ssh' and 'rest-server' does not allow specifying a single IPv6 address without a prefix.

Conditions:
Configure an 'allow-from' option specifying a single IPv6 address for either an 'ssh' or 'rest-server' object.

Workaround:
Specify the single IPv6 address with a 128-bit prefix (that is, with a trailing /128 added to the IPv6 address).

LRS-3371 (Moderate) Creating an SSL profile with a name length greater than 22 will not display available certificates with ? completion

Symptom:
When editing an SSL profile to add certificates and keys, at times the certificate or key object that is defined will not show up in the list of available options. This is seen when using the Tab key or ? to expand the list of available options for attaching a primary-certificate or private-key object.

Conditions:
This will occur when the length of the certificate or key object name is greater than 22 characters. This causes the CLI interpreter to fail to list all of the key or certificate options available.

Workaround:
You can manually enter the name of the certificate or key object, and it will apply correctly. This can be verified with the command "show ssl profile <name>" and check that the Cert Name or Key Name is correct.

LRS-3385 (Moderate) REST server process leaks memory after running a script that repeatedly logs in

Symptoms:
The system has a slow memory leak triggered by repeated client login requests to the REST server.

Conditions:
System is running with a configuration that has REST server access enabled. After processing client login requests to the REST server (on the order of several thousands of requests), the memory used by the REST server process goes up and doesn't appear to be released to the system after the client connections have closed.

Workaround:
None.

LRS-3389 (Moderate) virtual-ip statistics for HTTP response codes are incorrectly labeled as internal

Symptoms:
The 'show load-balancer statistics detailed' command shows statistics for HTTP response codes, as seen by a virtual IP. However, the name of the statistics have "Internal" in the name, such as 'httpInternalResp200'. These statistics actually count all HTTP response codes, not just internally generated responses, so the name of the statistic is incorrect.

Conditions:
The issue is always present when viewing 'show load-balancer statistics detailed'.

Workaround:
None.

LRS-3416 (Moderate) Show tech support detailed output is missing netstat details

Symptom:
Command timed out message instead of mbuf details in output of show command.

Conditions:
Execute "show tech support detailed".

Workaround:
The show tech command aggregates the output of several OS commands, any of which may time out individually due to load on the system. The issue is transient, and the command may be re-run to capture complete output

LRS-3520 (Moderate) ipv6 route command does not support mtu option

Symptom:
The "ipv6 route" command does not support the "mtu" option.

Condition:
Attempting to create a route with an associated MTU.

Workaround:
None.

LRS-3543 (Moderate) Redirect to the login page when click browser reload while the GUI is in the middle of an automatic reload.

Symptoms:
System redirects to the login page when you click the browser reload button while the GUI page is in the middle of an automatic reload.

Conditions:
1. Log in to the GUI and wait for the initial config data load to complete (that is, wait for the "Last update" message to appear and the "Reload Configuration" spinner to disappear).

2. Click the "Reload Configuration" button, and then click the browser "Reload Page" button while the "Reload Configuration" spinner is spinning.

Note that you are redirected to the login page, but the expected behavior is to return to the GUI home page.

Workaround:
Don't click the browser "reload" button while the "reload configuration" spinner is visible.

LRS-3549 (Moderate) REST: PUT to reset node to default fails with HTTP error 500

Symptoms:
Setting some REST API nodes to default value results in HTTP 500 error.

Conditions:
Sending an HTTP PUT command to some REST API nodes with default = 'true' results in the error.

Workaround:
Refer to the REST API documentation for the node to determine if the node can be set to default = 'true'. If it cannot, use the default value listed to explicitly set the node to its default. For 1.6.x, see https://docs.lineratesystems.com/100Release_1.6/250REST_API_Reference_Guide. For 2.x, see https://docs.lineratesystems.com/200Release_2.0/250REST_API_Reference_Guide.

Additional Information:
Using REST or the documentation, you can determine if it is possible to set the node to default = 'true'. Send a GET command to the node and see if default and defaultAllowed are set to = 'false'. If they are, you cannot use the REST API GET default = 'true' to set the node to its default value.

LRS-3562 (Moderate) Incorrect warning message on removing interface IP address after changing virtual-ip IP address

Symptoms:
A warning message is incorrectly displayed upon removal of an IP address from an interface. The warning message is similar to the following:
WARNING: deleting IP address associated with virtual-ip <virtual_ip_name>

Conditions:
This occurs with the following sequence of steps:
1. Start with IP address A configured on a virtual IP.
2. Change virtual IP's address from A to B.
3. Remove IP address A from an interface, and the warning message incorrectly gets displayed.

Workaround:
None needed. The message is only a warning.

LRS-3568 (Moderate) System redirects to a page whose content is "Internal Server Error".

Symptoms:
System redirects to a page whose content is "Internal Server Error" and subsequent login attempts will fail until the browser session cookie is manually deleted. This issue is reproducible in Chrome, but not in Firefox.

Conditions:
1. Log in to the GUI and wait for the refresh configuration spinner to stop spinning.
2. Click the Real Servers tab.
3. Click the Refresh Configuration button and while the refresh configuration spinner is still spinning, click the Log Out link.
4. Log in again.
The page whose only content is the text "Internal Server Error" displays.

Workaround:

  • Don't click the Log Out link while the system is in the middle of refreshing the configuration.
  • If you do get the "Internal Server Error" page, manually delete the connect.sid cookie before you try to log in again.

LRS-3597 (Moderate) MTU size does not change for IPv6 routes when changed on the physical interface

Symptom:
After changing interface MTU, the MTU value for the IPv6 routes for that interface still show the old MTU size.

Condition:
System configured for IPv6 interfaces and routes.

Workaround:
Statically configure the route with the ip route command.

LRS-3605 (Moderate) Link local addresses can be configured for interfaces even if they don't start with fe80

Symptom:
IPv6 addresses that are not link local (start with fe80) are accepted by the CLI and shown in the running configuration.

Condition:
None.

Workaround:
None. Take care to only configure link-local addresses when using the link-local keyword.

LRS-3633 (Moderate) IPv6 Routes do not check the destination before removing from the configuration

Symptom:
Removing an IPv6 route without the correct gateway address is allowed by the system. If any route with that destination network exists, it will be removed, regardless of whether the gateway given in the "no" command matched the gateway that was configured for that route.

Condition:
System is configured with static routes.

Workaround:
None

LRS-3635 (Moderate) Health monitor errors on valid response if connection is closed with TCP reset

Symptom:
HTTP health monitor does not show healthy real server, even though a correct and valid HTTP response was sent back to the health monitor request.

Conditions:
The server sending the HTTP response closes the connection with a TCP reset immediately after sending an HTTP response.

Workaround:
One of the following can resolve this issue:

  • Configure the HTTP server to close the connection gracefully after sending the HTTP response, instead of closing the connection with a TCP reset.
  • Configure the HTTP server to delay closing the connection with TCP reset by a few milliseconds after sending the HTTP response. However, this workaround will not work
    if the response body is delimited by the TCP connection closing (for example, "Connection: close" header).

LRS-3643 (Moderate) Error message: Bad message from route socket

Symptom:
Error message displayed in log files similar to the following:
LROS: Bad message from route socket

Condition:
Certain types of interface-related state changes, including CARP group state changes may cause this message.

Workaround:
None known.

LRS-3709 (Moderate) Real server pool and health monitor configuration can report the backup and primary real servers as "up" at the same time

Symptoms:
System reports that both the primary and secondary real servers are "up" at the same time.

Conditions:

  • Real server rs1 is the primary and real server rs2 is the secondary.
  • Health monitor hm1 is attached to rs1 and monitors rs1's IP address.
  • Health monitor hm2 is attached to rs2 and monitors rs2's IP address.
  • Health monitor hm3 is attached to rs2 and monitors rs1's IP address. hm3 has the same settings as hm1 (same values for "interval", "server-up", etc.), but it also has the "invert" option set.
  • User brings rs1 down, and the system correctly reports that rs1 is down and rs2 is up.
  • User brings rs1 back up, and the system incorrectly reports that both rs1 and rs2 are up.
  • The system can report that both the primary and backup real servers are "up" at the same time for the duration of one health-monitor interval.

Workaround:
The second health monitor (hm3) that is attached to the secondary real server (the health monitor that is supposed to be the invert of the primary server's health monitor) should have low values set for "interval", "server-up" and "server-down".

LRS-3750 (Moderate) REST: numChildren on a GET can sometimes return incorrect number of children for a node

Symptom:
The numChildren value returned in response to a GET in REST for some nodes in the /status hierarchy may sometimes be smaller than the actual number of children of that node.

Condition:
Certain nodes always return a numChildren value that is too low. No additional condition is required.

Workaround:
Perform a GET with the "op=list" query option and count the actual number of child nodes.

LRS-3751 (Moderate) PUT on certain REST nodes should return an error

Symptom:
The REST operation PUT is performed on a node where PUT is not a valid operation. The correct behavior is that an error should be issued for nodes that do not allow PUT on them.

Conditions:
This issue applies to many nodes, including some collection (organization) nodes, such as virtualServer.

Workaround:
None known.

LRS-3757 (Moderate) Attaching SSL profile to REST server after 'allow to' commands causes inconsistent state

Symptom:
The REST server may not listen on all configured IP addresses and ports. Additionally, the error message below will be present due to REST configuration:
ERROR: Unable to create socket: Address already in use

Conditions:
Multiple instances of 'allow to' are present in the REST server configuration, and the SSL profile is attached to the REST server after the 'allow to' configuration. Additionally, it may be that one of the 'allow to' lines must be 'allow to any'.

Workaround:
Configure either 'allow to any' or one or more other 'allow to' lines, but not both.

LRS-3791 (Moderate) Error messages in log on failover: write error: device busy

Symptom:
Error messages in the system logs similar to the following:
LROS: Initiation(102.250.1.155:16224) to Termination(101.0.180.1:63728 -> 101.0.130.4:9020) write error: Device busy

Condition:
Unknown, but may be related to layer 4 (TCP) load balancing and a failover event.

Workaround:
None known.

LRS-3793 (Moderate) Some virtual IPs don't come up correctly when set to use CARP IP

Symptom:
Virtual IP does not accept incoming traffic. Issuing 'show virtual-ip <name>' shows that the virtual IP is in a DOWN state due to IP address not being configured on the system, even though the IP address is correctly configured as a CARP IP.

Conditions:
Virtual IP is configured to use an IP address from a CARP group. Other unknown conditions are also required to trigger this issue.

Workaround:
The virtual IP can be brought up by configuring the virtual IP and issuing the following commands:
admin-status offline
admin-status online

LRS-3803 (Moderate) Entering IP route with next hop not directly connected does not warn

Symptom:
The system accepts, without warning, a static IP route that has a next hop that is not directly connected.

Conditions:
Configuring an IP route with the 'ip route' command where the next hop IP address is not contained within a directly connected network.

Workaround:
None.

LRS-3805 (Moderate) SNMP ifAlias not populated with interface description

Symptom:
The IF-MIB SNMP OID ifAlias does not contain a descriptive string, such as the configured interface description.

Condition:
The system is configured for SNMP.

Workaround:
None.

LRS-3811 (Moderate) Polling of CPU information via SNMP stopped retrieving information

Symptom:
System CPU load is not being retrieved via SNMP.

Conditions:
Polling SNMP OIDs related to system CPU utilization using an SNMP client, such as Cacti.

Workaround:
Use "top" from bash via the command line.

LRS-3829 (Moderate) tmr_tcp kernel thread - high CPU with many connections

Symptom:
Management of the system may become sluggish. Using "top -S" from bash, the tmr_tcp process is using a large percentage of CPU.

Conditions:
The system has a very large number of open TCP connections (greater than 1 million), and the traffic pattern causes a large number of delayed ACKs to be transmitted.

Workaround:
Note: Consider your changes carefully before using this workaround, because these commands can have a serious impact on your system. For 1.6.x, see https://docs.lineratesystems.com/100Release_1.6/200CLI_Reference_Guide/Configure_Commands/Load_Balancer_Mode_Commands. For 2.x, see https://docs.lineratesystems.com/200Release_2.0/200CLI_Reference_Guide/Configure_Commands/Proxy_Mode_Commands.

A CPU core can be devoted to the tmr_tcp process by decreasing the number of load balancer (version 1.6.x) or proxy (version 2.x) datapath processes by one and moving the tmr_tcp process.

LRS-3830 (Moderate) Extra VLAN encapsulation in port-mirrored packet capture

Symptoms:
On ix interfaces with no VLANs configured, packet captures can incorrectly include erroneous VLAN encapsulation.

Conditions:
Packet capture on an ix interface with no VLAN interface configured on top of it.

Workaround:
Configure a VLAN interface on top of the ix interface you want to sniff on. Packet captures on the underlying physical interface now show the correct VLAN encapsulation (if any).

LRS-3835 (Moderate) Out-of-order transmission of forwareded IP fragments on ix interface

Symptoms:
Forwarded fragments may be retransmitted out of order. Specifically, the first fragment may be delayed and transmitted after later fragments.

Conditions:
Forwarding fragments going out an Intel "ix" interface.

Workaround:
None.

LRS-3846 (Moderate) LROS installer exits when specifying a crash dump partition size larger than the disk

Symptom:
The LROS installer exits unexpectedly during the installation process.

Condition:
The user specifies a size for the dumpdev that is larger than the target disk.

Workaround:
None known. Enter a smaller size for dumpdev.

LRS-3862 (Moderate) IP routes with non-zero host portion should be rejected

Symptom:
A route with a destination network that contains non-zero host bits can be successfully entered using the "ip route" command. Routes like the two below are invalid and should be rejected by the system.
ip route 10.0.0.1/24 ix1
ipv6 route fd::2:0:0:7b1/64 ix1

Condition:
No additional conditions are required.

Workaround:
Compute the correct destination network for the route and re-enter the correct route. Corrections for the examples above are:
ip route 10.0.0.0/24 ix1
ipv6 route fd::/64 ix1

LRS-3892 (Moderate) recovery: restore command will only work if the target file exists in the default backup location (ie. /home/linerate/backups)

Symptoms:
Restore command fails when the target file isn't in the default backup location (/home/linerate/backups).

Conditions:
Try to restore a target file that doesn't exist in the default backup location (/home/linerate/backups).

Workaround:
First copy the target file to the default backups location and then issue the restore command.

LRS-3959 (Moderate) DSCP is inconsistent when Max-in-flight > 1

Symptoms:
DSCP is inconsistent.

Conditions:
Real server configured with max-in-flight >1.

Workaround:
Configuring DSCP with TCP multiplex enabled is not supported. The DSCP bits may not be correct in this case.

LRS-3997 (Moderate) Core files appear in /var/crash for the node process

Symptom:
Core files appear in /var/crash for the node process.

Conditions:
A PUT request is issued to the REST server containing a "data" field that
is an incorrectly formatted JSON object.

Workaround:
None.

LRS-4117 (Moderate) REST: /config/restServer/<name>/allow/from/<address> node is a string

Symptoms:
The /config/restServer/<name>/allow/from/<address> node is currently a string type.

Conditions:
None.

Workaround:
None.

Additional Information:
To be consistent with other IP address nodes, this node should be ip-addr-with-mask type.

LRS-4121 (Moderate) Can't distinguish between full and quick backup files in CLI or REST

Symptom:
Neither the show backup list command or /status/system/util/list node distinguish between a full or quick backup file.

Condition:
System backups have been created with the backup command.

Workaround:
When creating a backup, name the backup file with information that distinguishes the type of backup you are creating.

LRS-4640 (Moderate) REST API returns numChildren=0 for chainCertName with children

Symptom:
The JSON returned for the REST path /status/ssl/profile/<name>/chainCertName shows numChildren=0 even though there is at least one chain certificate installed and a corresponding node present.

Condition:
Any retrieval of the chainCertName REST API node.

Workaround:
Use the 'level=recurse' option on the query string to retrieve the actual list of installed chain certificates, and then count the number of direct child nodes returned.

LRS-4846 (Moderate) LACP interfaces are not always balanced

Symptom:
Traffic leaving the load balancer through an LACP port-channel is not evenly balanced between the physical interfaces that are members of that port-channel.

Conditions:
LACP is configured across a set of physical interfaces.

Workarounds:
None known.

Additional information:
Affects 1 G interfaces more severely than 10 G interfaces. The 10 G interfaces are typically within about 10% difference between physical interfaces.

LRS-4866 (Moderate) REST node: /config/snmp/servers/<name> has incorrect name

Symptoms:
The /config/snmp/servers/<name> node is incorrect.

Conditions:
None.

Workaround:
None.

Additional Information:
This node is not a node you create, as denoted by the <name> at the end, but is a fixed node. Additionally, there is just one SNMP server, meaning that server in the node path should not be plural.

LRS-4870 (Moderate) Scripting: A file descriptor opened by a script can be accessed by other scripts

Symptoms:
A file descriptor opened by a script can be accessed by other scripts.

Conditions:
Script foo opens a file and gets a file descriptor back. Script bar
does a read/write/close on this file descriptor.

Workaround:
Do not share file descriptors across scripts.

LRS-4872 (Moderate) LROS ping command does not set the TTL of the ping packet to what is requested

Symptom:
The TTL value of the ping command cannot be overridden by the the -m option for ping.

Condition:
Running ping command in bash.

Workaround:
None known.

LRS-5223 (Moderate) Inconsistent use of the "default" field when using the REST API to unset virtual-ip "IP Address" and "IP Range".

Symptoms:
Inconsistent use of the "default" field when using the REST API to unset virtual-ip "IP Address" and "IP Range".

Conditions:
When you want to unset a virtual-ip "IP Range" through the REST API, you have to submit a request with "default = 1" and "family = af-unspecified". But when you want to unset a virtual-ip "IP Address", you have to submit a request with "default = 0" and "family = af-unspecified".

Workaround:
None known.

Additional Information:

LRS-5233 (Moderate) Output from the forward-proxy "statistics detailed" show command contains invalid queue-related statistics.

Symptoms:
Output from the forward-proxy "statistics detailed" show command contains invalid queue-related statistics.

Conditions:
Create a forward-proxy object named "fp-1". Run the show command: "show forward-proxy fp-1 statistics detailed". Note that the output contains sub-sections entitled "Request Mgr Q Size:" and "Server Q Size:". These queue-related statistics blocks are not applicable to forward-proxy objects.

Workaround:
None known.

Additional Information:

LRS-5344 (Moderate) REST API doesn't perform the same input validations as the CLI.

Symptoms:
REST API doesn't perform the same input validations as the CLI.

Conditions:
For example, if you use the REST API to set a forward-proxy description value of "hello world", the double quotes will be preserved and when you execute "show run" at the CLI the output will show a description of ""hello world"" (ie. another set of wrapping quotes are added).

Workaround:
None known.

Additional Information:

LRS-5377 (Moderate) CLI uses port-channel to create a port channel and channel-group to configure the port channel

Symptoms:
The CLI command to create a port channel is interface port-channel <chan_num>. The command to configure the port channel is channel-group <chan_num>.

Conditions:
Creating and configuring port channels.

Workarounds:
None.

LRS-5378 (Moderate) Must use integer to create port channel, but pox to configure port channel

Symptoms:
CLI requires an integer to create a port channel, but requires pox (where x is the integer) to configure the port channel.

Conditions:
Creating and configuring port channels.

Workaround:
Use an integer to create the port channel, then use pox (where x is the integer) to configure the port channel.

LRS-5856 (Moderate) Excessive SSL messages printed to log file

Symptoms:
Messages similar to one or more of the following appear in the system logs:
LROS: Error accepting SSL connection: Encountered EOF before it was expected
LROS: Error accepting SSL connection: Dynamic exception type: lrs::io::ssl::SSLIOException
LROS: std::exception::what: no ciphers passed
LROS: std::exception::what: tlsv1 alert user cancelled
LROS: std::exception::what: tlsv1 alert internal error

Conditions:
SSL configured.

Workaround:
None known.

LRS-5947 (Moderate) Poor error message when upgrade runs out of disk space

Symptoms:
The upgrade command fails with an interaction similar to the following:
LROS# upgrade "file:///home/linerate/image_name.upg"
Upgrade requires a reload when completed.
Continue? [yes/no]: yes
ERROR: Error upgrading: Running /boot/install failed.
Unexpected error upgrading LROS

Conditions:
The system is low on disk space. Typically, an upgrade requires about 2GB of free disk space.

Workaround:
Look for error messages in /var/log/controller.messages during the upgrade process. This should give more information on the failure. Removing un-needed files can allow the upgrade to complete successfully.

LRS-5954 (Moderate) System does not respond to any HTTP requests

Symptoms:
System stops responding to HTTP requests.

Conditions:
1) Real server attached to the virtual server must have an SSL profile name attached to it.
2) The attached SSL profile must not be configured on the system.
3) The real server must not have an health monitor attached to it.
4) The virtual server must support TCP multiplexing.

After the virtual server processes a few HTTP requests in the above configuration, unconfigure the SSL profile attached to the real server. The virtual server will stop processing any new HTTP requests.

Workaround:
Attach a health monitor monitor to the real server or
offline the real server administratively and bring it back online using the commands "admin offline" and "admin online"

Additional Details:
After queuing up around 10000 requests per system process, the system responds with HTTP 503 messages for all subsequent requests.
This issue can also happen in non-tcp-multiplex mode, but it requires the unconfiguration of the SSL profile attached to the real server to happen after the systm starts sending 503 response code HTTP responses.

LRS-5962 (Moderate) Proxy does not honor real server weights properly

Symptoms:
System does not load balance the HTTP requests properly based on real server weights using the weighted least connections algorithm.

Conditions:
When tcp-multiplex and weighted least connections are configured on virtual servers, either of the following conditions may cause this:

  • If real servers have "is-proxy" set to off and the virtual servers have "forward-connect-requests" set to true, HTTP CONNECT requests are dropped.
  • Certain transmission errors to the real server that result in the proxy closing the connections.

Workaround:
Administratively offline and online the real servers using the "admin offline" and "admin online" configuration commands.

Additional details:

LRS-6040 (Moderate) Client timeout with large HTTP objects with Weighted Least Connection Load Balancing algorithm with no tcp-multiplex

Symptom:
HTTP clients with short timeouts may time out waiting for responses with objects greater than 8 k in size.

Condition:
The system is configured as a reverse proxy (load balancing), the virtual server is using the weighted least connections load balancing algorithm, and tcp-multiplex is disabled on the virtual server.

Workaround:
Enable tcp-multiplex or change the load balancing algorithm.

LRS-6160 (Moderate) Show CARP lists incorrect IP on "Master is ..." line

Symptoms:
The show carp <group> commands lists the incorrect IP address in the "Master is ..." line.

Conditions:
At least two CARP groups and virtual IP configured.

Workaround:
None.

LRS-6258 (Moderate) Removing the IP address does not display the message - "WARNING: deleting IP address associated with virtual-ip <vipname>"

Symptoms:
The message "WARNING: deleting IP address associated with virtual-ip vip1" is not displayed.

Conditions:
1. Delete the IP address used by virtual IP (vip1) configured on an interface.
2. Add the IP address used by vip1 on a different interface.
3. Delete the IP address from the interface again.

Workaround:
None.

Additional Information:
This issue does not impact the functionality of the system.

LRS-6355 (Moderate) Cannot create or remove a virtual IP named "range" using the CLI

Symptom:
You cannot create or remove a virtual IP named "range" using the CLI.

Condition:
Try to create or remove a virtual IP named "range."

Workaround:
Create or remove with the REST API.

Additional Information:
For a virtual IP in the CLI, range is reserved word for the virtual-ip command.

LRS-6449 (Moderate) Virtual server in TCP multiplex mode does not process any client requests

Symptoms:
LineRate does not process any HTTP requests.

Conditions:

  • A virtual server must have real servers attached to it
  • A virtual server must have a virtual IP attached to it
  • All the real servers attached to the virtual server must have "max-connections" set to 0 (or default).
  • "tcp-multiplex" must be enabled on the virtual server the first time after the real servers are attached

Workaround:
Set the "max-connections" on the real servers to a value greater than zero.
Enable "tcp-multiplex" again on the virtual server

Additional Details:
If "tcp-multiplex" is enabled prior to setting the "max-connections" on the real servers, an error message is generated indicating that "max-connections" must be set for "tcp-multiplex". In that case, set the "max-connections" to a non-zero value and then enable "tcp-multiplex" one more time.

LRS-6492 (Moderate) REST: Virtual IP stats for both forward and reverse proxies shown in clientNoVserver stats

Symptoms:
Virtual IP stats that relate to both forward and reverse proxies are in the clientNoVserver stats node.

Conditions:
None.

Workaround:
None. Node to be renamed clientNoProxy in the future.

LRS-6536 (Moderate) Need to clean-up the persistence-related warning messages on pxeboot images.

Symptoms:

  • The warning messages that are presented when you execute the "write" or "copy" commands on a pxeboot image are split across multiple lines instead of being written as a single, coherent warning message.
  • The pxeboot image's persistence-related warning messages contain partial file path references instead of complete file path references. For example, they contain file paths like "/startup-config" instead of "/home/linerate/data/startup-config".

Conditions:

  • If you enter the command "write mem" on a pxeboot image, you'll get this output:
    WARNING: File written to a volatile storage location: /startup-config
    WARNING: This will not persist across a reboot. Please copy to a
    WARNING: non-volatile location when able.
    Building configuration...
    [OK]
  • If you enter the command "copy running-config tmp.txt" on a pxeboot image, you'll get this output:
    WARNING: File written to a volatile storage location: /tmp.txt
    WARNING: This will not persist across a reboot. Please copy to a
    WARNING: non-volatile location when able.
    Copying file...done.

Workaround:
None known.

Additional Information:

LRS-6730 (Moderate) Selecting upgrade from installation menu exits to shell if no existing installation is found

Symptoms:
Installer program exits to shell prompt when "Upgrade" option is selected, but an existing installation is not found.

Conditions:
In installer program, "Upgrade" option is selected, but an existing installation is not found.

Workaround:
Do not attempt to upgrade a system without an existing installation. If you mistakenly exit to the shell, type "exit" or press CTRL-D to exit the shell and return to the installer or "reboot" to reboot the system.

Additional Information:

LRS-7306 (Moderate) Cannot log in to system to use CLI or REST

Symptom:
In some rare instances, the system can get into a state where it
does not accept new CLI or REST client requests.

Condition:
This can happen when previous configuration commands take a long
time to service. These situations usually occur when:
a. the system is under heavy load.
b. the system has a really large configuration and is syncing it to
data path processes (which happens on system startup or when
a data path process starts up).
c. Both of the above.

Workaround:
None known.

LRS-7739 (Moderate) no attach real-server on deleted but attached server doesn't tab complete

Symptoms:
Tab completion does not work on objects that have been deleted.

Conditions:
Configure real server real-foo, attach it to virtual IP vip-foo, then delete real server real-foo ("no real-server real-foo"). In vip-foo mode, try "no attach real-server rea[TAB]." Tab completion does not work.

Workaround:
Type the full name of the real-server or health-monitor in the "no attach" command.

Additional Information:

LRS-8412 (Moderate) Packets dropped counter is the same for both oce interfaces in system

Symptoms:
The dropped packets counter in the output of the show interface command for oce interfaces is the same value for all interfaces installed in the system

Conditions:
System running in a server with oce interfaces installed.

Workaround:
None known.

LRS-8413 (Moderate) Output errors on oce interfaces after CARP switchover

Symptoms:
Output errors reported in show interface output on Emulex (oce) interfaces after CARP switchover.

Conditions:
System running on a server that has Emulex interfaces installed in it and CARP configured.

Workaround:
None known.

Additional Information:

LRS-8414 (Moderate) Packets dropped counter rolls over at low values for oce interfaces

Symptoms:
The packets dropped stat reported in show interface output on Emulex (oce) interfaces rolls over to 0 at low values.

Conditions:
System running on a server that has oce interfaces installed in it that is receiving traffic that is being dropped by the interface.

Workaround:
None known.

Additional Information:

LRS-8424 (Moderate) TCP Retransmission Errors

Symptoms:
Running system commands in bash such as sysctl -a or show tech-support with the CLI, which take a long time to complete, cause the system to fail over due to CARP keep-alive-timeout. This timeout out is short-lived, so the master changes back quickly, which causes TCP retransmissions.

Conditions:
System with CARP configured and a user entering system or CLI commands that take a long time to complete.

Workaround:
None.

Additional Information:
If the system is busy or has a lot connections active, the commands will take a long time to complete causing this error. This is a transient error that only happens while the command is executing or shortly after it completes.

LRS-8930 (Moderate) Error: SNMP actual retrieval of routing table: Cannot allocate memory

Symptoms:
Error: snmpd[14786]: actual retrieval of routing table: Cannot allocate memory message shown in the /var/log/messages file.

Conditions:
System configured for SNMP and client doing SNMP walk.

Workaround:
None.

Additional Information:

LRS-9271 (Moderate) Custom error object handling inconsistent with node.js

Symptoms:
Thrown exceptions show up in the syslog as '[object Object]', and their source lines are printed (sometimes truncated), rather than the full, formatted exception message.

Conditions:
Custom exceptions (of the form 'throw

{name:"foo", message:"bar"}

;' are being used.

Workaround:
Use the 'new Error("foo","bar");' syntax instead.

Additional Information:
The node.js JavaScript interpreter parses the custom error messages correctly when run in non-interactive mode. In interactive mode, the custom errors are unrecognized. The V8 JavaScript interpreter does not parse such custom objects as exceptions, but prints them as objects with their source line. LineRate's interpreter is consistent with V8 and the node.js interactive interpreter.

LRS-9334 (Moderate) scheduler sometimes has error: "cpuset: setaffinity: No such process"

Symptoms:
The "scheduler cpu manual" command returns an error similar to the following:

ERROR: cpuset error for thread 100503 (controller/controller): cpuset: setaffinity: No such process

Conditions:
Using "scheduler cpu manual" on a multithreaded process that starts and stops threads frequently.

Workaround:
If the command is repeated, it may complete successfully.

Additional Information:
This error does not cause any other undesired behavior other than the message itself. It simply indicates that a transient thread of the multithreaded process has exited prior to setting the CPU affinity for that thread.

LRS-9336 (Moderate) Garbled syslog output from scripting engine

Symptoms:
Error messages from multiple data path (lb_http) processes are interleaved, causing garbled lines in /var/log/controller.messages.

Conditions:
A script with a runtime or syntax error is loaded in the scripting engine.

Workaround:
The properly formatted error message is available from "show script <script>".

LRS-9576 (Moderate) CARP ads not received for ix VLAN interface on backup

Symptoms:
Both units in CARP group are operating as the master, leading to traffic problems as an upstream switch or router may not pass traffic properly through the CARP group.

Conditions:
CARP is configured on a VLAN interface attached to an ixgbe physical interface.

Workaround:
None. Reboot of CARP backup system is required.

Additional Information:
The trigger for this problem is unknown. However, CARP switchovers were being deliberately forced, and in other cases, the interface driver has been observed to perform a reset when going "admin-status up". Although, in those cases, the interface could be manually reset to correct the CARP issue, whereas in this case no reset path was identified short of a system reboot.

LRS-9585 (Moderate) GUI doesn't poll the backend system on a regular basis.

Symptoms:
GUI fails to display configuration changes that were made outside the GUI (ie. changes that were made through the CLI or REST API).

Conditions:
Log in to the GUI, click on the "Real Servers" tab in the configuration table and apply the filter "name = rs-x". Verify no matches are found and then remove the filter. Go to the CLI and add a new real-server named rs-x. Go back to the GUI and re-apply the filter "name = rs-x". Verify the GUI still returns no matches. Remove the filter, click the "Refresh Configuration" button and then re-apply the filter. Note that this time the GUI returns one match (ie. the GUI recognizes the config change after you clicked the "Refresh Configuration" button).

Workaround:
Periodically refresh the configuration by clicking the "Refresh Configuration" button.

Additional Information:
Prior versions of the GUI would poll the backend system at regular intervals to pick up config changes and update the dashboard's configuration widgets accordingly. Prior versions of the GUI would also allow this polling feature to be enabled/disabled with a checkbox on the "Information" dialog.

LRS-9805 (Moderate) Scripting HTTP.ClientRequest doesn't support hostname option

Symptoms:
Scripting HTTP client requests with the 'hostname' option ignore the option, and always request to host '127.0.0.1'.

Conditions:
A script that makes a request using the 'hostname' option:

var http = require('http');
var req = http.get({ hostname: '3.4.5.6',
                     port: 80,
                     path: '/index.html' },
                   onResponseCb);

Workaround:
Use the 'host' property instead:

var http = require('http');
var req = http.get({ host: '3.4.5.6',
                     port: 80,
                     path: '/index.html' },
                   onResponseCb);

Additional Information:

LRS-10373 (Moderate) Sample file names used in the upgrade command help message end with ".img" instead of ".upg".

Symptoms:
Sample file names used in the upgrade command help message end with ".img" instead of ".upg".

Conditions:
At the CLI, issue the command "upgrade ?" to see the the upgrade command help message. Note that the sample file names used in the help message end with ".img" instead of ".upg".

Workaround:
n/a

Additional Information:

LRS-10461 (Moderate) GUI fails to apply additional formatting to configuration table cells that contain delimited list values.

Symptoms:
GUI fails to apply additional formatting to configuration table cells that contain delimited list values, which makes these cells unreadable without manual intervention.

Conditions:
Create a virtual-server and attach three real-servers. Log in to the GUI, click the "Virtual Servers" tab on the configuration table control and find the newly created virtual-server. Note that the contents of the "Real Servers" column is a comma-separated list of real-server names and is unreadable without manual intervention (ie. in the absence of text-wrapping or some other formatting assistance, you're forced to resize the column to read its contents).

Workaround:
Manually resize the affected columns to make their content readable.

Additional Information:

LRS-10483 (Moderate) System allows users to edit public dashboards they don't own.

Symptoms:
System allows users to edit public dashboards they don't own.

Conditions:
System contains two users: user-1 and user-2. Log in as user-1, create a public dashboard and add a line chart widget to the dashboard. Log out as user-1 and log in as user-2. Add, remove and edit widgets on the public dashboard that you created as user-1. Log out as user-2 and log back in as user-2. Note that none of the changes you made as user-2 were persisted.

Workaround:
You should treat public dashboards as read-only and avoid trying to edit them.

Additional Information:

LRS-10485 (Moderate) System only allows one dashboard to be deleted at a time.

Symptoms:
System only allows one dashboard to be deleted at a time.

Conditions:
Open the Dashboards dialog and create two new dashboards by clicking the "+" button twice. Now try to delete both of the newly created dashboards by clicking their associated checkboxes and then clicking the "-" button. Note that the system only allows one dashboard to be selected at a time.

Workaround:
When you want to delete multiple dashboards, you have to delete them one at a time.

Additional Information:

LRS-10610 (Moderate) System refuses to accept the opening paren when you type a filter that uses the attribute name "Session Cache Size (bytes)".

Symptoms:
System displays the error message "Enter a different attribute name." when you type a filter that uses the attribute "Session Cache Size (bytes)".

Conditions:
Log in to the GUI and click the "SSL Profiles" tab in the configuration table control. Click into the filter textbox, type the filter condition "Session Cache Size (bytes) > 0" and hit "Enter". Note that the filter condition that's actually applied is "Session Cache Size bytes) > 0" (ie. note that the system refuses to accept the opening paren before "bytes").

Workaround:
Instead of typing the full attribute name into the filter textbox, you can just select the attribute name from the type-ahead suggestions list.

Additional Information:

LRS-10633 (Moderate) System fails to clearly communicate chart widget's chart type.

Symptoms:
System fails to clearly communicate chart widget's chart type

Conditions:
Create a line chart for the real-server connections statistic "Server Open Connections" (allow "Objects" to default to "All"). Create an area chart for the exact same statistic. Note that both the line and area charts appear identical on your dashboard and the system provides no way to distinguish between the two.

Workaround:
None known.

Additional Information:

LRS-10640 (Moderate) Monitoring charts fail to display coordinate values when you hover over a data point.

Symptoms:
Monitoring charts fail to display coordinate values when you hover over a data point.

Conditions:
Hover over a data point on a monitoring chart and note that the system fails to display a tooltip that shows the data point's coordinate values.

Workaround:
Estimate the x-axis value from a visual inspection of the chart. The y-axis value can be found in the line label text that appears in the top right corner of the chart when you hover over a data point.

Additional Information:

LRS-10722 (Moderate) Set default=false ignored for script attributes when override value is the same as the default value.

Symptoms:
With REST API path /status/script/<script name>/restartMode or /status/script/<script name>/adminStatus, the default setting remains set to True after setting a value equal to the default (0).

Conditions:
Set REST API paths /status/script/<script name>/restartMode or /status/script/<script name>/adminStatus, to a value equal to the default.

Workaround:
Either ignore the default setting or set to non-default value (1 in this case), and then set back to 0 to clear the default setting.

LRS-11348 (Moderate) ix interface takes 5 seconds to come out of reset

Symptoms:
When reconfiguring an Intel "ix" network interface, the system may take up to approximately 5 seconds to start sending and receiving traffic. After traffic starts moving through the ix network interface, performance is normal.

Conditions:
This brief delay in traffic happens when the network interface configuration is changed, for example, when the LRO and TSO flags are enabled or disabled.

Workaround:
None.

Additional Information:

LRS-11352 (Moderate) ix interface traffic byte-count stats are off by 6%

Symptoms:
Intel "ix" network traffic statistics (bytes sent and received) may be off by 5-10%.

Conditions:
When bytes are sent and received by an Intel "ix" network interface, the bytes reported by the "netstat" command may be incorrect by 5-10%.

Workaround:
None.

Additional Information:

LRS-11821 (Moderate) System will pass through HTTP Upgrade: headers it does not understand, corrupting the channel

Symptoms:
System will forward all Upgrade headers and then terminate the server connection when data is sent, if the upgrade was to a non-HTTP protocol (for example, HTTPS, FTP, etc.).

Conditions:
Client request contains an Upgrade: header requesting an upgrade to a non-plain text HTTP protocol. System forwards this header to a server. The server understands the upgrade request and switches protocols.

Workaround:
Configure strip-request-headers on the virtual server to remove all Upgrade headers from requests.

LRS-11943 (Moderate) System fails to sort newly copied objects into the correct row.

Symptoms:
System fails to sort newly copied objects into the correct row.

Conditions:
This issue only occurs if there are more objects than can fit on a single page of the configuration table (ie. if the configuration table has to add a vertical scrollbar to show all the objects) and if the configuration table hasn't been manually sorted already (ie. you haven't clicked any column headers yet). If both of these conditions exist, then if you copy an existing object and give it a name that is almost identical to the original (eg. copy real-server "rs-http-1" to "rs-http-2"), you'll notice that the newly created object does not automatically appear next to the original object. You have to manually sort on the "Name" column to make the newly created object appear next to the original.

Workaround:
After copying an object, click on the "Name" column header to manually sort on that column.

Additional Information:

LRS-11965 (Moderate) System fails to deselect configuration table rows when a filter is removed.

Symptoms:
System fails to deselect configuration table rows when a filter is removed.

Conditions:
Apply a filter (for example "name != ''"), then select one or more of the resulting rows. Remove the filter and note that the same number of rows are still selected. The expected behavior is that no rows remain selected when the filter is removed.

Workaround:
You can manually remove

Additional Information:

LRS-13138 (Moderate) Category names in the filter feature's autocomplete suggestions list are not sorted alphabetically.

Symptoms:
Category names in the filter feature's autocomplete suggestions list are not sorted alphabetically.

Conditions:
Navigate to the "Virtual Servers" tab on the configuration table control. Type "a" into the filter textbox and note that the "SCRIPT" category appears below the "VIRTUAL IP" category in the resulting autocomplete suggestions list.

Workaround:
None known.

Additional Information:

LRS-13145 (Moderate) Chart statistics list is difficult to read when object names are long enough to overflow a single line in the statistics list.

Symptoms:
Chart statistics list is difficult to read when object names are long enough to overflow a single line in the statistics list.

Conditions:
Add a real-server named real-server-default-vals. Create a line chart, set the object type to "Real Server", set the object name to "real-server-default-vals". Add the "HTTP Requests" statistic "Total HTTP Server Requests" and the "Connections" statistic "Server Open Connections". Note that the first statistic overflows into the second line in the "Statistics" section of the chart settings dialog and the second statistic is then overlaid on top of this second line, which makes the list of associated statistics difficult to read.

Workaround:
None known.

Additional Information:

LRS-13146 (Moderate) Dropdown controls on the chart settings dialog don't display properly in Chrome on Windows and Ubuntu.

Symptoms:
Dropdown controls on the chart settings dialog don't display properly in Chrome on Windows and Ubuntu.

Conditions:
Open the chart settings dialog in Chrome on a Windows or Ubuntu machine and note that bottom parts of the characters in the dropdown controls are truncated.

Workaround:
None known.

Additional Information:

LRS-13150 (Moderate) pico editor has strange backspace behavior when using tmux

Symptoms:
When editing a file via the included editor in the system, characters disappear and reappear.

Conditions:
Editing existing lines of code when using tmux version 1.7

Workaround:
Do not use tmux version 1.7. Use tmux version 1.6.

Additional Information:

LRS-13170 (Moderate) Dashboard management dialog indicates that public dashboards are supported.

Symptoms:
Dashboard management dialog indicates that public dashboards are supported.

Conditions:
Open the dashboard management dialog and note that it contains a column of checkboxes entitled "Public". Public dashboards are not supported in this release, so this column of checkboxes should be removed.

Workaround:
None known.

Additional Information:

LRS-13201 (Moderate) Memory and CPU utilization spike when creating and deleting many real servers (or virtual servers)

Symptoms:
Memory and CPU utilization spike when creating or deleting many real servers and virtual servers.

Conditions:
Any time many real servers or virtual servers are being created or deleted.

Workaround:
None.

Additional Information:

LRS-13213 (Moderate) GUI fails to recognize object name if the name wasn't specified in the object creation request's payload.

Symptoms:
GUI configuration table contains rows with empty "Name" cells.

Conditions:
Use the REST API to create a new configuration object, but omit the 'data' key from the json data that is sent with the object creation request.

Workaround:
Specify the object name in the 'data' key in the json data that is sent with the object creation request.

Additional Information:

LRS-13243 (Moderate) REST: some nodes below /config/ssl/profile/<name> are not used

Symptoms:
Unused REST nodes are available.

Conditions:
None.

Workaround:
Do not use these nodes:
/config/ssl/profile/<name>/keyLifeTime
/config/ssl/profile/<name>/sslVersionsOpenSslFormat
/config/ssl/profileBase/<name>/keyLifeTime
/config/ssl/profileBase/<name>/sslVersionsOpenSslFormat

Additional Information:

LRS-13505 (Moderate) GUI fails to treat real-server base "Description" attribute as read-only

Symptoms:
GUI allows the real-server base "Description" attribute to be edited.

Conditions:
Create a real-server base object, click on its "Description" cell in the configuration table and note that the GUI opens the "Description" editor. Real server base objects do not support the "Description" attribute, so the GUI should not open an editor when this cell is clicked.

Workaround:
None known.

Additional Information:

LRS-13512 (Moderate) npm tmp directory contents not cleaned up in some cases

Symptoms:
Some temporary files are left in /home/linerate/data/scripting/tmp directory that do not get cleaned up
when they are no longer needed.

Conditions:
Seen occasionally when npm installs fail.

Workaround:
Remove files in this directory manually.

Additional Information:

LRS-13675 (Moderate) Default cell editor hangs if user tries to save with "Use Default" after a failed attempt to save with "Set Locally".

Symptoms:
Default cell editor hangs if user tries to save with "Use Default" after a failed attempt to save with "Set Locally".

Conditions:

  • open the real-server "Max Connections" cell editor
  • select the "Set Locally" radio button, enter a value of 1234 and click "Save"
  • re-open the cell editor, clear the textbox and click "Save"
  • note that an editor error message is displayed ("Enter a valid number")
  • select the "Use Default" radio button and click "Save"
  • note that the editor error message remains and the editor fails to save the change and close

Workaround:
Close and re-open the cell editor. Note that the "Set Locally" field contains the cell value you want to unset. Select "Use Default" and click "Save".

Additional Information:

LRS-13947 (Moderate) System hangs during boot after changing /home/linerate/data to be a symlink to /tmp

Symptoms:
Unable to SSH into system or connect via REST API. System is hung while booting just after a message is display on the console 'Clearing /tmp'. The serial console is still accessible.

Conditions:
After entering the bash prompt from lros_shell and changing /home/linerate/data to be a symlink to /tmp instead of pointing the physcial disk. System is reloaded from CLI or power-cycled and is in the process of rebooting.

Workaround:
Configure /home/linerate/data to point to physical disk storage. If /home/linerate/data must point to temporary (or temporarily unreachable) storage, the system startup may hang. Connect to the serial console and press 'Ctrl-C' to terminate the startup. However, you cannot log in to the product. You need to boot from an ISO (such as a LiveCD), manually mount the physical disk on which LROS is installed, and configure /home/linerate/data to point to a persistent (not /tmp) location.

Additional Information:
This is a non-standard deployment would be manually configured. You cannot configure the system in this manner from lros_shell. This item informational in case someone wants to map parts of the directory structure to either /tmp storage, or perhaps some offbox location (for example, NFS).

LRS-14261 (Moderate) Licensing error message: "Couldn't resolve host login.f5.com"

Symptoms:
User cannot get a trial license from the license server because the login failed with the error message "Couldn't resolve host 'login.f5.com'"

Conditions:
System that is unlicensed and is trying to install a license via the F5 license sever.

Workaround:
Ensure there is connectivity to the DNS servers that are configured. DNS servers can be seen with the "show ip dns" command. One reason for this error is that the system may not be configured with a route that allows it to reach the DNS servers. For how to configure for licensing, see https://docs.lineratesystems.com/200Release_2.0/100Getting_Started_Guide/130Configuring_Licensing.

Additional Information:

LRS-14295 (Moderate) GUI displays a "Create Error" message instead of redirecting to login page (on an expired session).

Symptoms:
After clicking a tab, object icon, or configuration reload button, the GUI displays a "Create Error" dialog.

Conditions:
The "Create Error" dialog is displayed after the session idle timeout period has expired.

Workaround:
Log in to the application again by clicking the browser reload button or by clicking the log out link located at the top of the application. Enter a valid user name and password in the login page to start a new session.

Additional Information:

LRS-14296 (Moderate) GUI displays poor error message when duplicate address is used for start and end address in the virtual-ip "IP Address Range" attribute.

Symptoms:
GUI displays the "Save Error" message "hostname, nor servname provided, or not known".

Conditions:
The same IP address is used for the "start" and "end" addresses when configuring the virtual-ip "IP Address Range" attribute.

Workaround:
n/a

Additional Information:

LRS-14373 (Moderate) Warning message: Error retrieving value of object

Symptoms:
A warning message similar to the following appears in the log files:
Error retrieving value of object /script/<script-name>/attached/proxyRequestListener/add during sync: Internal error: Node Has No Data
Error retrieving value of object /script/<script-name>/attached/proxyRequestListener/del during sync: Internal error: Node Has No Data
Error retrieving value of object /script/<script-name>/signalRuntimeError during sync: Internal error: Node Has No Data

Conditions:
During system startup or when data path processes are started.

Workaround:
None.

LRS-14549 (Moderate) umbrella - LineRate Manager displays the error message: "Unable to communicate with host x"

Symptoms:
LineRate Manager displays the error message "Cannot communicate with host x".

Conditions:
The conditions that generate this error are unknown, but it seems to occur most regularly while performing forward proxy configuration operations through LineRate Manager.

Workaround:
None known.

Additional Information:
LineRate Manager displays this error message when it times out waiting for a response from the REST server.

LRS-14559 (Moderate) GUI fails to display an error message for invalid ssl-profile "Cipher List" values.

Symptoms:
GUI fails to display an error message for invalid ssl-profile "Cipher List" values.

Conditions:
GUI user configures an invalid value (eg. "abc") for the ssl-profile "Cipher List" attribute.

Workaround:
n/a

Additional Information:

LRS-14796 (Moderate) Script editors don't work well when user connects via mosh/tmux.

Symptoms:
When using "source edit vim" or "source edit pico" to edit a script, the editor becomes unusable. For example, cursor keys insert lines with numbers instead of moving the cursor.

Conditions:
Script editor is invoked from a 256-color terminal, with TERM set incorrectly, such as when running in a tmux session.

Workaround:
Don't use mosh/tmux to access the CLI and edit scripts, or set $TERM to the correct value before logging in. For tmux or screen: export TERM=screen-256color

Additional Information:

LRS-14913 (Moderate) When GUI is refreshed (CTRL-r), tabs of whiteboard and config panes are reset to default

Symptoms:
Whiteboard and config panes are reset to default tab when LineRate Manager GUI page is reloaded by user.

Conditions:
User presses CTRL-r, F5, or click the reload icon in browser while viewing the LineRate Manager GUI.

Workaround:
Reselect the correct tabs, or don't refresh.

Additional Information:

LRS-15139 (Moderate) Scripting fs module has no chmod method

Symptoms:
JavaScript exception when calling fs.chmodSync.

Conditions:
A script makes a call to fs.chmodSync.

Workaround:
Fix file permissions from the bash shell using the chmod command.

Additional Information:

LRS-17560 (Moderate) HTTP/1.1 client requests served by HTTP/1.0 real servers are not always closed

Symptoms:
Some client connections are not closed immediately.

Conditions:
An HTTP/1.1 client makes a request with a body, and the back-end server behind the proxy is HTTP/1.0

Workaround:
None known.

Additional Information:

LRS-17953 (Moderate) Requests/sec not split evenly across multiple identical clients

Symptoms:
Requests/sec not split evenly across multiple identical clients.

Conditions:
System has a license limiting HTTP requests/sec, and client load exceeds the limit. Each client uses a dedicated virtual IP to isolate traffic type and measure distribution of the license capacity.

Workaround:

Additional Information:
All clients are able to sustain requests through the system. However, the client sending requests to the virtual IP with the lowest IO address appears to receive 50% of the licensed capacity. Remaining clients appear to evenly split the remaining 50% of licensed capacity.

LRS-17954 (Moderate) System limits transactions/sec to under the value entitled by the license

Symptoms:
System limits transactions/sec to under the value entitled by the license.

Conditions:
Script uses .newRequest() API to redirect a request between two proxies.

Workaround:
None

Additional Information:
Requests flowing through the data path, when caught by a script, should only be counted once by the licensing feature. Only new requests or connections initiated by a script should be counted explicitly and added to the total requests being captured by licensing.

LRS-17959 (Moderate) GUI doesn't perform the same license file validations that the CLI performs.

Symptoms:
GUI doesn't perform the same license file validations that the CLI performs.

Conditions:

  • GUI fails to display an error message when it imports an empty license file or a license file that's larger than 100mb.
  • GUI fails to display an error message when it imports a license file that uses an unsupported character set (eg. utf-16).

Workaround:
n/a

Additional Information:

LRS-18310 (Moderate) show snmp-server always requires host

Symptoms:
The "show snmp-server" command always requires "host" to be a valid command. The host setting is not currently in use.

Conditions:
None.

Workaround:
Use "show run" to see the snmp-server settings that configured on the system.

Additional Information:

LRS-18312 (Moderate) In LineRate Manager, copying a file-based script makes a duplicate entry pointing to the same file

Symptoms:
Copied script file actually points to the original file.

Conditions:
Use LineRate Manager to copy a file-sourced script.

Workaround:
Copy the script to the clipboard with the mouse, then create a new script and paste it in.

Additional Information:

LRS-21127 (Moderate) Non-UTF-8 characters do not display correctly in show certificate output and generate an error message

Symptoms:
The "show certificate" and "show certificate bundle" commands generate "libprotobuf ERROR ..." message in syslog. Additionally, the show output doesn't display non-UTF-8 characters correctly.

Conditions:
Configuration where a certificate whose parsed data has any non UTF-8 characters is used to configure a LineRate certificate or certificate bundle.

Workaround:
None

Additional Information:
The certificate is accepted and works correctly.

LRS-21559 (Moderate) Some [no] syntax examples are not valid when used without no

Symptoms:
Some syntax examples in the CLI help that use [no] in the example are only valid as the no form of the command. The form of the command without no is not valid.

Conditions:
Using the CLI ? help.

Workaround:
In general, configuration commands that refer to a named object require the inclusion of the object name.

Additional Information:

LRS-21618 (Moderate) node.js's util.inspect does not print all request object fields

Symptoms:
The util.inspect() method may not print all fields of a ServerRequest object.

Conditions:
Calling util.inspect(request):

var util = require('util');
vs.on('request', function (servReq, servResp, cliReq) {
  console.log('ServerRequest:', util.inspect(servReq));
  // Other processing
});

Workaround:
The 'url', 'method', 'httpVersion', 'httpVersionMajor', and 'httpVersionMinor' properties are present, but must be enumerated separately:

var util = require('util');
vs.on('request', function (servReq, servResp, cliReq) {
  console.log('ServerRequest:', util.inspect(servReq));
  console.log('servReq.url:', servReq.url);
  console.log('servReq.method:', servReq.method);
  console.log('servReq.httpVersion:', servReq.httpVersion);
  console.log('servReq.httpVersionMajor:', servReq.httpVersionMajor);
  console.log('servReq.httpVersionMinor:', servReq.httpVersionMinor);
  // Other processing
});

Additional Information:
The 'headers' and 'connection' properties are present and will be printed by util.inspect().

LRS-21627 (Moderate) Time-based system proxy stats are not the sum of per-proxy stats

Symptoms:
Time-based stats (/sec, /sec 1 Min Avg, /sec 5 Min Avg) in the output of "show proxy statistics detailed" are not equal to the sum of individual (per-proxy) stats in the output of "show virtual-server <name> statistics detailed". The same is true for forward-proxy.

Conditions:
The output appears correct when only a single proxy is configured, but does not appear correct when multiple proxies are configured.

Workaround:
None

Additional Information:
Time-based statistics conform to the Unix convention of "exponential decay." Meaning each sample in a window of time is weighted, with samples at the beginning of the window receiving less weight than samples at the end of the window. The system statistic is calculated by taking the sum of the per-proxy dividends but with its own independent (exponential decay) divisor algorithm.

LRS-23740 (Moderate) Use of __scriptname global triggers a spurious warning in the LineRate Manager script editor

Symptoms:
A yellow warning icon appears next to lines containing the __scriptname global in the LineRate Manager script editor.

Conditions:
User is editing a script in the LineRate Manager script editor.

Workaround:
Ignore this warning, it is inaccurate.

Additional Information:

LRS-24011 (Moderate) Hostname is "LROS" in logs, even after changing the hostname

Symptoms:
Hostname in logs is usually the string "LROS," even after a different hostname has been configured.

Conditions:
Always

Workaround:
You can manually restart rsyslogd after every boot and hostname change. Note that log messages generated while rsyslogd is restarting will be lost:

bash "sudo service rsyslogd restart"

Additional Information:

LRS-24765 (Moderate) Memory leak when fastpiping server request with response option and 'response' listener

Symptoms:
Proxy processes use larger amounts of memory as reported by top. Eventually, the processes may be killed by the out-of-memory killer and some customer traffic will be temporarily disrupted.

Conditions:
A script registers a listener for the ClientRequest's response event and calls ServerRequest.fastPipe() with an auto-fastPipe ServerResponse target, for example:

vs.on(request, function(servReq, servResp, cliReq) {
  // BAD: Can only register for 'response' OR fastPipe() with auto-fastPipe on response, not both.
  cliReq.on('response', function(cliResp) {
    // If you are auto-fastPiping the response (like below), this is never called.
    console.log('Caught response:', cliResp);
  });
  servReq.bindHeaders(cliReq);
  servReq.fastPipe(cliReq, { 'response': servResp });
});

Workaround:
Only register for the response event OR auto-fastPipe the response, not both. You can check if a response listener is registered when auto-fastPiping:

vs.on('request', function(servReq, servResp, cliReq) {
  ...
  if (cliReq.listeners('response').length) {
    // There is a listener, we can't auto-fastPipe
    servReq.fastPipe(cliReq);
  } else {
    // There is no listener, we can auto-fastPipe
    servReq.fastPipe(cliReq, { 'response' : servResp });
  }
});

You can also call cliReq() as a function, which will do the same thing:

vs.on('request', function(servReq, servResp, cliReq) {
  ...
  // Call cliReq() as a function: auto-fastPipe the response if there are no response listeners.
  cliReq();
});

Additional Information:
Auto-fastPiping the response prevents the response event from firing, per the API documentation. The never-called response callback is useless. The resolution of this bug will prevent the memory leak but the response callback will still not be called, and will be useless.

LRS-25265 (Moderate) HTTP statistics graphing options are not greyed out in TCP VIPs in GUI

Symptoms:

HTTP statistics graphing options show as selectable options in drop-down menu of graphing tool for TCP VIP objects

Conditions:

User uses the graphing feature of the GUI to select a TCP type statistic, such as a TCP VIP.

Workaround:

Ignore the HTTP options for TCP objects.

Additional Information:

LRS-26017 (Moderate) Denial of service attack when using OpenSSL with ECDH (CVE-2014-3470)

CVE
CVE-2014-3470: OpenSSL TLS clients enabling anonymous ECDH ciphersuites are subject to a denial of service attack.

Conditions:
Anonymous ECDH ciphersuites are enabled and a vulnerable OpenSSL is installed on the system.

Workaround:
Disable anonymous ECDH ciphersuites

Additional Information:
OpenSSL 0.9.8 pre "za", OpenSSL 1.0.0 pre "m", OpenSSL 1.0.1 pre "h" are vulnerable.
More details on this vulnerability can be found at http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3470

LRS-26018 (Moderate) Local users may obtain ECDSA nonces via side-channel attack (CVE-2014-0076)

CVE
CVE-2014-0076: Local users may obtain ECDSA nonces via a FLUSH+RELOAD cache side-channel attack

Conditions:
Vulnerable version of OpenSSL is on the system.

Workaround:
None

Additional Information:
Systems using OpenSSL before version 1.0.0m are vulnerable. LineRate has been updated to a more recent version of OpenSSL.
More details on this vulnerability can be found at http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0076

LRS-26216 (Moderate) Scripting performance drop in scripting response processing

Symptoms:
LineRate Scripting performance drops 13% from previous version only when the script processes HTTP responses.

Conditions:
Upgrade from 2.2.4 to 2.2.5 with a script installed that catches responses from real-server sent back to the DUT.

Workaround:
None

Additional Information:

LRS-26399 (Moderate) npm fails to install package with "unable to find location of package from install output"

Symptoms:
Receive the error "unable to find location of package from install output" when attempting to install an npm module.

Conditions:
The version string contains a . in the version modifier. For example, the . after rc1 in the version:
some-pkg@1.0.0-rc1.4

Workaround:

  • Install a different version that does not contain a . in the version modifier.
  • You can execute the following command to install the requested version. However, it will not honor npm registry settings, including SSL certificates, and the installed version will not show up in show scripting packages user until reboot:

lros# bash
[user@lros ~]$ sudo /usr/local/bin/bash -c "cd \
/home/linerate/data/scripting \
 && HOME=/home/linerate/data/scripting \
PATH=/usr/linerate/node/scripting/bin:$PATH  \
/usr/linerate/node/scripting/bin/npm install <PACKAGE>"

Additional Information:

LRS-26400 (Moderate) Output of "show carp" incorrectly indicates master has no IP address

Symptoms:
The "show carp" command states that master has no IP address.

Conditions:
Remove the IP address of the interface that contains CARP subconfig. Then re-add the IP address. Output of "show carp" command now reflects that master has no IP address.

Workaround:
Unknown.

Additional Information:
This is a cosmetic bug and CARP will continue to function correctly.

LRS-27167 (Moderate) PXE loader fails to mount NFS share if the IP address of the NFS server is different than the TFTP server

Symptoms:
Diskless software (PXE boot) deployment of an image fails to boot.

Conditions:
LineRate deployed as a KVM guest using PXEBOOT.

Workaround:
Ensure that the next-server and root-path both have the same IP address in them.

Additional Information:
This happens when the NFS server and the TFTP server are on different machines. There are messages on the console of the VM as it boots that indicate that the ARP entry for the NFS server (the one specified in root-path) is not found.

LRS-28843 (Moderate) CVE-2014-3512 (openssl - SRP buffer overrun)

Symptoms:
SRP buffer overrun (CVE-2014-3512)

Conditions:
SRP cipher is disabled by default in the controller; however, it can be enabled by scripting.

Workaround:
Disable SRP cipher.

Additional Information:
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3512

LRS-28844 (Moderate) CVE-2014-3511 (openssl - OpenSSL TLS protocol downgrade attack)

CVE:
CVE-2014-3511 (openssl - OpenSSL TLS protocol downgrade attack)

Conditions:
A flaw in the OpenSSL SSL/TLS server code causes the server to negotiate TLS 1.0, instead of higher protocol versions when the ClientHello message is badly fragmented. This allows a man-in-the-middle attacker to force a downgrade to TLS 1.0, even if both the server and the client support a higher protocol version, by modifying the client's TLS records.

Workaround:
None

Additional Information:
http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-3511

LRS-28846 (Moderate) CVE-2014-3508 (OpenSSL Information leak in pretty printing functions)

CVE:
CVE-2014-3508 (OpenSSL Information leak in pretty printing functions)

Conditions:
Possible buffer overflow attack in LineRate certificate show commands.

Workaround:
None

Additional Information:

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3508

LRS-28925 (Moderate) Running LineRate ova in VMware workstation causes the host (PC) to hang

Symptoms:
Host PC hangs when starting LineRate in VMware Workstation.

Conditions:
Import LineRate into VMware Workstation.
Run the LineRate instance without adding a serial port.

Workaround:
Add a serial port before running the LineRate instance.

Additional Information:

LRS-29422 (Moderate) Popup values for statistic graphs need greater precision for values < 1

Symptoms:
When using LineRate Manager with either a line chart or area chart configured to display latency statistics, values less than 0.05 seconds will be formatted as 0.0 in the chart legend. The graph line or area value is plotted correctly, but the legend value can read as 0.0 for values less than 50 milliseconds.

Conditions:
Using LineRate Manager line or area charts to graph latency values, including:
Client Transaction Latency
Server Transaction Latency

Workaround:
Read the value from the graph Y axis and do not rely on the legend value.

Additional Information:

LRS-29476 (Moderate) Unsupported ECC curves can be used

Symptoms:
Unsupported ECC curves could be used.

Conditions:
SSL profile is configured to use ECC-based ciphers.

Workaround:
Configure a supported ECC curve in all SSL profiles that use ECC ciphers.

Additional Information:
The list of curves that are not supported, and will be disabled are:
sect113r1 SECG curve over a 113 bit binary field
sect113r2 SECG curve over a 113 bit binary field
sect131r1 SECG/WTLS curve over a 131 bit binary field
sect131r2 SECG curve over a 131 bit binary field
sect163k1 NIST/SECG/WTLS curve over a 163 bit binary field
sect163r1 SECG curve over a 163 bit binary field
sect163r2 NIST/SECG curve over a 163 bit binary field
sect193r1 SECG curve over a 193 bit binary field
sect193r2 SECG curve over a 193 bit binary field
sect233k1 NIST/SECG/WTLS curve over a 233 bit binary field
sect233r1 NIST/SECG/WTLS curve over a 233 bit binary field
sect239k1 SECG curve over a 239 bit binary field
sect283k1 NIST/SECG curve over a 283 bit binary field
sect283r1 NIST/SECG curve over a 283 bit binary field
sect409k1 NIST/SECG curve over a 409 bit binary field
sect409r1 NIST/SECG curve over a 409 bit binary field
sect571k1 NIST/SECG curve over a 571 bit binary field
sect571r1 NIST/SECG curve over a 571 bit binary field
c2pnb163v1 X9.62 curve over a 163 bit binary field
c2pnb163v2 X9.62 curve over a 163 bit binary field
c2pnb163v3 X9.62 curve over a 163 bit binary field
c2pnb176v1 X9.62 curve over a 176 bit binary field
c2tnb191v1 X9.62 curve over a 191 bit binary field
c2tnb191v2 X9.62 curve over a 191 bit binary field
c2tnb191v3 X9.62 curve over a 191 bit binary field
c2pnb208w1 X9.62 curve over a 208 bit binary field
c2tnb239v1 X9.62 curve over a 239 bit binary field
c2tnb239v2 X9.62 curve over a 239 bit binary field
c2tnb239v3 X9.62 curve over a 239 bit binary field
c2pnb272w1 X9.62 curve over a 272 bit binary field
c2pnb304w1 X9.62 curve over a 304 bit binary field
c2tnb359v1 X9.62 curve over a 359 bit binary field
c2pnb368w1 X9.62 curve over a 368 bit binary field
c2tnb431r1 X9.62 curve over a 431 bit binary field
wap-wsg-idm-ecid-wtls1 WTLS curve over a 113 bit binary field
wap-wsg-idm-ecid-wtls3 NIST/SECG/WTLS curve over a 163 bit binary field
wap-wsg-idm-ecid-wtls4 SECG curve over a 113 bit binary field
wap-wsg-idm-ecid-wtls5 X9.62 curve over a 163 bit binary field
wap-wsg-idm-ecid-wtls10 NIST/SECG/WTLS curve over a 233 bit binary field
wap-wsg-idm-ecid-wtls11 NIST/SECG/WTLS curve over a 233 bit binary field

LRS-31484 (Moderate) Unnecessary "Disabling profiler" message displays after npm command

Symptoms:
Unnecessary message about disabling profiler is displayed when an npm is installed.

Conditions:
Issue the CLI command:
scripting npm install <package>

Workaround:
Ignore the message.

Additional Information:

LRS-34677 (Moderate) Error message "LROS: Health SysdbRemote get ERROR..." seen in the system logs when running "show real-server <rs-name> health" command

Symptoms:
Error messages, such as "LROS: Health SysdbRemote get ERROR...", appear in the system logs when running the "show real-server <rs> health" CLI command.

Conditions:
The system is experiencing high load on the CPU where the health monitor process is running.

Workaround:
Avoid running the "show real-server <rs> health" command as much as possible.

Additional Information:

LRS-38494 (Moderate) CLI displays unexpected/inconsistent output when invalid license file is installed.

Symptoms:
CLI displays a licensing info table with a single row even though there is no active/valid license installed.

Conditions:
Install an invalid license file and then issue the command "show licensing brief". Note that the resulting table contains a single row (when you might have expected that it would be an empty table).

Workaround:
None known.

Additional Information:

LRS-38714 (Moderate) CLI accepts a username UID of -1

Symptoms:
System creates new users with negative UID values.

Conditions:
Enter a CLI config command that creates a user with a negative UID (eg. "username foo secret xyz uid -1"). Note that LROS accepts the command and doesn't present any warning/error messages. Now isssue the "show running config" command and note that the negative UID is displayed in the output.

Workaround:
Don't create users with negative UID values.

Additional Information:

LRS-38731 (Moderate) CLI - real-server accepts (and displays) negative values for all its timeout attributes

Symptoms:
System allows user to specify negative values for real-server timeout values (eg. "Keepalive Timeout", "Response Timeout", etc.).

Conditions:
Create a new real-server with service type "http" and set its "Response Timeout" to -1. Note that the system accepts this invalid timeout value and fails to display any warning/error messages.

Workaround:
Don't use negative values when setting real-server timeouts.

Additional Information:

LRS-38744 (Moderate) "no ip" command for snmp-server hosts fails to remove the IP address

Symptoms:
Unable to remove IP address associated with an SNMP host from the running configuration.

Conditions:
Configure an IP address for an SNMP host. Then try to remove it using the command

no ip <ipaddress>

Workaround:
Remove all configuration associated with the SNMP host via:

snmp-server
no host <host-name>

Additional Information:

LRS-42225 (Moderate) SNMP table index does not return all values

Symptoms:
SNMP does not return all objects in a table, per the MIB definition

Conditions:
Any SNMP operation, such as get, get-next, or get-bulk, used to obtain MIB objects from LRS MIBs.

Workaround:
The index is embedded as part of the OID when walking a table. It can be discovered by taking the last component of the OID (after the last '.').

Additional Information:
snmptranslate -m ALL -Tp .1.3.6.1.4.1.33661.2.2.1.2.1
+--lrsVipStatEntry(1)
Index: lrsVipName
+-- R- String lrsVipName(1)
Textual Convention: LrsVipName
Size: 1..100
+-- R- EnumVal lrsVipIpAddressType(2)
Textual Convention: InetAddressType
Values: unknown(0), ipv4(1), ipv6(2), ipv4z(3), ipv6z(4), dns(16)
+-- R- String lrsVipIpAddress(3)
Textual Convention: InetAddress
Size: 0..255

From above output we see that lrsVipName OID contains VIP names that serve as index for remaining OIDs in the table.

Doing snmpwalk against lrsVipName returns nothing:
snmpwalk -v 2c -c public 192.168.217.140 .1.3.6.1.4.1.33661.2.2.1.2.1.1
SNMPv2-SMI::enterprises.33661.2.2.1.2.1.1 = No Such Object available on this agent at this OID

snmpwalk -v 2c -c public -m ALL 192.168.217.140 lrsVipName
LRS-LB-MIB::lrsVipName = No Such Object available on this agent at this OID

Doing snmpwalk on lrsVipStatTable does return values for all other OIDS, with vip names as index.

snmpwalk -v 2c -c public -m ALL 192.168.217.140 .1.3.6.1.4.1.33661.2.2.1.2.1
LRS-LB-MIB::lrsVipIpAddressType."vip1" = INTEGER: ipv4(1)
LRS-LB-MIB::lrsVipIpAddress."vip1" = Hex-STRING: 0A 01 01 07
LRS-LB-MIB::lrsVipPort."vip1" = Gauge32: 80
LRS-LB-MIB::lrsVipBytesRx."vip1" = Counter64: 0
LRS-LB-MIB::lrsVipBytesTx."vip1" = Counter64: 0
LRS-LB-MIB::lrsVipBytesRxPerSec."vip1" = Counter64: 0
LRS-LB-MIB::lrsVipBytesTxPerSec."vip1" = Counter64: 0
LRS-LB-MIB::lrsVipConnClientOpened."vip1" = Counter64: 0
LRS-LB-MIB::lrsVipConnClientOpenedPerSec."vip1" = Counter64: 0

LRS-43232 (Moderate) LineRate does not reboot when instance runs out of swap space

Symptoms:
During the course of operation a LineRate instance becomes unresponsive and unreachable through its data interfaces.

Conditions:
The LineRate instance had been receiving traffic or scripting is enabled with at least one running script ("admin-status online"). Activity must be occurring on the instance to allow for processes to consume memory.

Workaround:
After LineRate becomes locked due to an out of memory condition, LineRate cannot recover and must be rebooted manually.

Additional Information:
If remote logging is enabled messages indicating processes were killed due to "out of swap space" may be produced:
Mar 19 16:45:03 host-XXX kernel: pid 2114 (node), uid 2001, was killed: out of swap space
Mar 19 16:45:04 host-XXX LROS: ProcManager: #REST exited: exited due to signal 9. Restarting.
Mar 19 16:45:04 host-XXX kernel: pid 1021 (node), uid 0, was killed: out of swap space
Mar 19 16:45:04 host-XXX LROS: Disabling profiler because SIGPROF handler is already in use.
Mar 19 16:45:05 host-XXX LROS: ProcManager: #REST exited: exited due to signal 9

LRS-44967 (Moderate) Scripting: Redis messages recieved in Nodej.s contain incorrect data

Symptoms:
Received message events in node-redis in a pub/sub configuration contain incorrect message data. This affects redis@v0.8.2 through redis@0.11.

Conditions:
Message events in redis contain errors in the callback function of node-redis. Subsequent messages after a given erroneous message will be incorrect.

Workaround:
None supported.

Additional Information:

Note: While upgrading redis to 0.12 ("scripting npm install redis") resolves this issue, it is not a fully tested configuration.

LRS-45934 (Moderate) Segmentation fault when script is halted while scripting TCP connection is in progress

Symptoms:
The lb_http process exits due to a segmentation fault, as seen in the system logs error: ProcManager: lb_http exited: exited due to signal 6 (core dumped).

Conditions:
A script is connecting to an external service using net.connect() at the same time that it is halted.

Workaround:
None

Additional Information:

LRS-46088 (Moderate) Some versions of node fail to accept DSA signatures that others do

Symptoms:
LineRate Scripting rejects some DSA/ECDSA signatures that it used to accept when using crypto.verify()

Conditions:
The DSA/ECDSA signatures have DER-encoded components with the most significant bit set.

Workaround:
Prepend the DER component with an extra 0 byte. Here is an example of this modification:
https://github.com/mscdex/ssh2-streams/commit/d4ecde6cd9af0710d4a65f90f04f047cf98ff6d9

Additional Information:
This issue is caused by a change in behavior for OpenSSL 1.0.1k. It now rejects signatures that were accepted in previous versions. See https://github.com/openssl/openssl/blob/OpenSSL_1_0_1-stable/CHANGES#L172 for more details. Versions of Node.js that use OpenSSL 1.0.1k or later have the same issue.

LRS-46659 (Moderate) Allocation error with this message "Scripting Fatal Error: Evacuation Allocation failed - process out of memory"

Symptoms:
This error is seen in the system logs: "Scripting Fatal Error: Evacuation Allocation failed - process out of memory" and lb_http processes periodically restart.

Conditions:
Scripts must be configured and set to admin-status online at least once.

Workaround:
Manually running "debug proxy js-run-gc" may allow for garbage collection to occur, but no known deterministic solution exists.

Additional Information:
By setting the script to online, the Scripting Tool allocates pages out of the code space portion of its heap. After this space is filled, it cannot garbage collect and emits a fatal error.

LRS-47068 (Moderate) "show running config" command does not display certain HTTP virtual IP properties

Symptoms:
The show running config command does not display certain HTTP virtual IP properties.

Conditions:
The HTTP virtual IP properties are configured using the REST API, without setting the service type of the virtual IP to HTTP.

Workaround:
Set the service type of the virtual IP to HTTP.

Additional Information:

LRS-47075 (Moderate) Startup snmp traps dropped when trap interface is configured with DHCP

Symptoms:
The synthetic linkStatus and carpStatus trap messages sent at system startup are not sent.

The system logs similar to:
ERROR: Sending SNMP Trap failed: snmptrap: Failure in sendto (Sub-id not found: (top) -> ifOperStatus) (Network is unreachable)
ERROR: Sending SNMP Trap failed: snmptrap: Failure in sendto (Sub-id not found: (top) -> lrsCarpInterface) (Network is unreachable)

Conditions:
The interface used to send the traps is configured to obtain its IP address using DHCP. This interface used is dictated by the route to the configured address of the host in the snmp-server configuration mode.

Workaround:
Configure the interface with a static IP address.

Additional Information:

LRS-47113 (Moderate) Scripts receive unexpected results or ENOTFOUND errors when script contains undefined domain arguments

Symptoms:
Scripts receive unexpected results or ENOTFOUND errors for queried hosts.

Conditions:
This can arise if undefined variables are used in the dns.resolve family of functions. This will only occur if the "hostname" argument is an undefined variable. An undefined variable may be interpreted as the string "undefined" and queries will be sent out to the configured servers. A DNS server may have an answer for the "undefined" host or may respond with ENOTFOUND.

Workaround:
Instrument your JavaScript code to analyze if your variables are undefined. If so, be sure to initialize them or test for undefined variables, before calling the dns.resolve functions.

Additional Information:

LRS-48065 (Moderate) REST API for persistSourceIp/cache returns incorrect type - "null"

Symptoms:
REST API returns incorrect type for persistSourceIp/cache object under virtual-server

Conditions:
Using REST API, retrieve the status node for the object
/status/app/proxy/virtualServer/<virtual-server-name>/serviceHttp/persistSourceIp/cache

Workaround:
None

Additional Information:

LRS-49448 (Moderate) Unable to add ix interfaces to channel group

Symptoms:
The channel group sub-configuration command under an ix interface fails with an error message:
"ERROR: Unable to set channel-group number: Internal error: No additional information available"

Conditions:
Interface does not have an MTU configured

Workaround:
Configure the MTU for all interfaces to be added to the port-channel via "mtu <number> under each interface sub-configuration.

Additional Information:

LRS-50105 (Moderate) 'no ip address dhcp' does not remove the interface IP address after 'shutdown'

Symptoms:
Issuing command 'no ip address dhcp' after 'shutdown' does not remove the DHCP-obtained IP address of the interface.

Conditions:
Configure the interface to enable DHCP. After it has a valid IP address, issuing the following commands should - but doesn't - remove the ip address from the interface:
(config:interface:em0)# shutdown
(config:interface:em0)# no ip address dhcp

Workaround:
Remove the IP address using 'no ip address dhcp' before shutting down the interface.

Additional Information:
Tested in VM environment.

LRS-50169 (Moderate) TCP proxy closes client connections early

Symptoms:
TCP proxy closes client connections early

Conditions:
LineRate running as a TCP reverse proxy and
Client sends a TCP FIN on the connection to indicate that it will not be sending any data, but will accept any data from the server

Workaround:
None

Additional Information:

Supported Downgrade Version

LineRate® supports using the upgrade command to install any newer version of system software. The only supported previous version that may be installed with the upgrade command is 2.5.1.