Did you configure Upstream/ Downstream (Max Bandwidth Provided by ISP)? The router is using these values to calculate how much bandwitch to reserve for the services in the rules and how much will have for the rest of the trafic.
By default these values are on 10000Kbps. Which means that after excluding the rate control max value (2 times 96), your bandwidth for the rest of the trafic should be max 9, something Mbps.
If you havn't done that, please try to configure the max bandwidth from the ISP and share the speedtest result.
I know I'm replying to an old thread but I have experienced the same result when I configure bandwidth management on the RV320. I called Cisco support and found out that this is a limitation of the RV320:
"High Priority domain shares about 60% of the bandwidth, Low Priority domain shares 10% and Middle Priority (outside rules) shares 30%." These values are hardcoded and cannot be manipulated.
You can try testing with MTU size and change the negotiation of both ports WAN and LAN from Auto to manual. Try with symetric configuration - 100 WAN and 100 LAN, or if both connections are on gigabit 1000/1000.
But in your case, I would suggest to call the support line and an engineer to investigate more closely the configuration, in order to find a solution.
Article ID:5748 Use TheGreenBow VPN Client to Connect with RV34x Series
Router Introduction A Virtual Private Network (VPN) connection allows
users to access, send, and receive data to and from a private network by
means of going through a public or share...
Article ID:5728 Configure a Teleworker VPN Client on the RV34x Series
Router Objective The Teleworker VPN Client feature minimizes the
configuration requirements at remote locations by allowing the device to
work as a Cisco VPN hardware client. When the T...
Article ID:5708 Configure the LAN and DHCP Settings on the RV34x Series
Router Objective A Local Area Network (LAN) is a network limited to an
area such as a home or small business that is used to interconnect
devices. LAN settings can be configured to li...