Queues and Data Rate Management

Document revision 1.7 (01-Aug-2003)
This document applies to MikroTik RouterOS v2.7

Table of Contents

Summary

Queuing is a mechanism that control data rate allocation, delay variability, timely delivery, and delivery reliability. The MikroTik RouterOS supports the following queuing mechanisms:

The queuing can be used for limiting the data rate for certain IP addresses, protocols or ports. The queuing is performed for packets leaving the router through a real interface. It means that the queues should always be configured on the outgoing interface regarding the traffic flow. There are two additional virtual interfaces which are used to limit all the traffic coming to (global-in) or leaving (global-out) the router regardless of physical interface.

Specifications

Packages required : system
License required : Basic (DEMO license is limited to 4 queues)
Home menu level : /queue
Protocols utilized : None
Hardware usage: significant

Related Documents

Software Package Installation and Upgrading
IP Addresses and Address Resolution Protocol (ARP)
Packet Filter (Firewall) and Network Address Translation (NAT)

Description

Classless Queues

There are four types of simple queues implemented in RouterOS: PFIFO, BFIFO, SFQ and RED. With Bytes First-In First-Out (BFIFO) and Packets First-In First-Out (PFIFO) packets are served in the same order as they are received. The only difference between BFIFO and PFIFO is that PFIFO has a length measured in packets, BFIFO in bytes. Generally, you do not want to use BFIFO or PFIFO as traffic shapers. It's better to use them just for statistics as they are pretty fast. The only exception is when you are running out of resources with RED and/or with complicated queue tree.

Stochastic Fair Queuing (SFQ) cannot limit traffic at all. Its main idea is to equalize sessions (not computer traffic, but session traffic, it is sometimes mentioned as SFQ drawback) when your link is completely full. It works in round-robin fashion, giving each session a chance to send sfq-allot bytes. Its algorithm can distinguish only 1024 sessions, and that is why several sessions can be treated as one. Each sfq-perturb seconds it drops internal table mixing all the connections and creates a new table. As it is very fast, you may want to use it as a child queue.

The normal behavior of queues is called tail-drop. Tail-drop works by queuing up to a certain amount, then dropping all traffic that 'spills over'. Random Early Detection (RED is also known as Random Early Drop because it actually works that way) statistically drops packets from flows before it reaches its hard limit. This causes a congested backbone link to slow more gracefully. It starts dropping packets when threshold reaches red-min-threshold mark randomly with increasing probability as threshold rising. Maximum probability is used when traffic reaches red-max-threshold mark. Then packets are simply thrown away. burst parameter is the number of packets allowed to burst through the interface when the link is empty (generally value of (min+min+max)/3 works fine). The minimum value that can be used here is equal to the value of red-min-threshold.

Classful Queues

Classful queues are very useful if you have different kinds of traffic which should have different treatment. Generally, we can set only one queue on the interface, but in RouterOS even simple queues (known as classless queues) are attached to the main (attached to the root, which represent real interface) Hierarchical Token Bucket (HTB) and thus have some properties derived from that parent queue. With classful queues it is possible to deploy hierarchical queue trees. For example, we can set a maximum data rate for a workgroup and then distribute that amount of traffic between the members of that group as we can do with simple queues attached to the main HTB, but with upper limit.

Each queue represents a virtual interface with the allowed data rate. It can be borrowed from sibling queues (queues that are children of one queue) when max-limit is greater than limit-at. If so, the queue would use over the allocated data rate whenever possible. Only when other queues are getting too long and a connection is not to be satisfied, then the borrowing queues would be limited at their allocated data rate.

When a parent is allowed to send some amount of traffic, it asks its inner queues in order of priority (priorities are processed one after another, from 1 to 8, where 1 means the highest priority). When the a queue reached its limit-at value, its priority is not to be taken in account, such a queue will be less-prioritative than the ones not reached this limit.

Information Rates and Contention Ratios

Quality of Service (QoS) means that router should prioritize and shape network traffic. QoS is not so much about limiting, it is more about providing quality. The main terms used to describe the level of QoS for network applications are:
  • CIR (Committed Information Rate) - the guaranteed data rate. It means that traffic not exceeding this rate should always be delivered.
  • MIR (Maximal Information Rate) - the maximal data rate router will provide.
  • Contention Ratio - the ratio to which the defined data rate is shared between users (i.e., data rate is allocated to a number of subscribers). For example, the contention ratio of 1:4 means that the allocated data rate may be shared between no more than 4 users.
  • Priority - the order of importance in what traffic will be processed. You can give priority to some traffic in order it to be handeled befor some other traffic.

    MikroTik RouterOS may be used to provide CIR and MIR with some contention level and priority. Here we will talk in terms of queues (which represent either real or virtual interface) and classes (children of a queue; each class has an another queue attached to it).

  • limit-at property is used to specify CIR. If the queue will be able to provide that data rate, it will (i.e, the parent queue (and the link the router is connected to) should be able to provide the total data rate equal or greater that the sum of all CIRs the queue should satisfy in order to quarantee these CIRs). CIRs will be satisfied in order of their priority.
  • max-limit property is used to specify MIR. If the queue has satisfied all the CIRs and it is able to provide some additional data rate, it will try to distribute that additional data rate between all its classes regardless of their priorities and not exceeding their MIRs.
  • Filters in RouterOS are very powerful and flexible. Providing Contention Ratio is only one application of what they can do. Using firewall mangle you can mark some a number of hosts with a flow-mark, so the data rate allocated for that mark will be shared between these hosts.

    Virtual Interfaces

    In addition to real interfaces, there are two vitrual interfaces you can attach queues to:
  • global-out - represents all the output interfaces in general. Queues attached to it applies before the ones attached to a specific interface.
  • global-in - represents all the input interfaces in general (INGRESS queue). Please note that queues attached to global-in applies to incomming traffic, not outgoing. global-in queueing is taking place just after mangle and before DST-NAT.

    Universal Client and Simple Queues

    Universal client should catch traffic when it leaves the router - in order to be able to function properly. But interface queues are made to catch the traffic last. Due to this obvious conflict, a not expected behavior (which cannot be considered as a bug, it is just the way interface queues work) is observed: while firewall filters and virtual interface queues (global-in and global-out) are working with the translated addresses, simple queues attached to the interface Universal Client is functioning on 'see' the original, not translated, IP address. So if it is necessary to match the download (outgoing to the client) traffic by the address Universal Client assigns to the client, either global-out queue or queue trees should be used.

    Queue Types

    Submenu level : /queue type

    Description

    The queue types are used to specify some common argument values for queues. There are four default built-in queue types: default, ethernet-default, wireless-default, and synchronous-default. The built-in queue types cannot be removed.

    Property Description

    name (name)- name for the queue type
    kind (pfifo | bfifo | red | sfq; default: pfifo) - kind of the queuing algorithm used:
  • pfifo - Packets First-In First-Out
  • bfifo - Bytes First-In First-Out
  • red - Random Early Detection
  • sfq - Stochastic Fair Queuing
    bfifo-limit (integer; default: 15000) - BFIFO queue limit. Maximum packet number that queue can hold
    pfifo-limit (integer; default: 10) - PFIFO queue limit. Maximum byte number that queue can hold
    red-limit (integer; default: 60) - RED queue limit
    red-min-threshold (integer; default: 10) - RED minimum threshold
    red-max-threshold (integer; default: 50) - RED maximum threshold
    red-burst (integer; default: 20) - RED burst
    sfq-perturb (integer; default: 5) - how often to change hash function
    sfq-allot (integer; default: 1514) - amount of data in bytes that can be sent in one round-robin round

    Notes

    For small limitations (64kbps, 128kbps) RED is more preferable. For larger speeds PFIFO will be as good as RED. RED consumes much more memory and CPU than PFIFO & BFIFO.

    Example

    To add red queue type with minimum threshold of 0, without any burst and named CUSTOMER-def: 
    [admin@MikroTik] queue type> add name=CUSTOMER-def kind=red \
\... red-min-threshold=0 red-burst=0
[admin@MikroTik] queue type> print
  0 name=default kind=none bfifo-limit=15000 pfifo-limit=10 red-limit=60
    red-min-threshold=10 red-max-threshold=50 red-burst=20 sfq-perturb=5
    sfq-allot=1514

  1 name=ethernet-default kind=none bfifo-limit=15000 pfifo-limit=10
    red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20
    sfq-perturb=5 sfq-allot=1514

  2 name=wireless-default kind=sfq bfifo-limit=15000 pfifo-limit=10
    red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20
    sfq-perturb=5 sfq-allot=1514

  3 name=synchronous-default kind=red bfifo-limit=15000 pfifo-limit=10
    red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20
    sfq-perturb=5 sfq-allot=1514

  4 name=CUSTOMER-def kind=red bfifo-limit=15000 pfifo-limit=10 red-limit=60
    red-min-threshold=0 red-max-threshold=50 red-burst=0 sfq-perturb=5
    sfq-allot=1514

[admin@MikroTik] queue type>

    Interface Default Queues

    Submenu level : /queue interface

    Property Description

    interface (name) - interface name
    queue (name; default: default) - default queue for the interface

    Example

    To change the default queue type to wireless-default for the prism1 interface: 
    [admin@MikroTik] queue interface> print
  # INTERFACE                             QUEUE
  0 ether1                                default
  1 prism1                                default
[admin@MikroTik] queue interface> set prism1 queue=wireless-default
[admin@MikroTik] queue interface> print
  # INTERFACE                             QUEUE
  0 ether1                                default
  1 prism1                                wireless-default
[admin@MikroTik] queue interface>

    Configuring Simple Queues

    Submenu level : /queue simple

    Description

    Simple queues can be used to set up data rate management for the whole traffic leaving an interface, or for certain source and/or destination addresses. For more sophisticated queue setup use the queue trees described further on.

    Property Description

    name (name; default: queue1) - name of the queue
    src-address (IP address/mask) - source IP address
    dst-address (IP address/mask) - destination IP address
    interface (name) - outgoing interface of the traffic flow
  • global-in - match all incomming traffic
  • global-out - match all outgoing traffic
    limit-at (integer; default: 0) - allocated stream data rate (bits/s)
  • 0 - no limit
    queue (name; default: default) - queue type. If you specify the queue type other than default, then it overrides the default queue type set for the interface under /queue interface
    priority (1...8; default: 8) - flow priority, 1 is the highest
    max-limit (integer; default: 0) - maximum stream data rate (bits/s)
  • 0 - no limit

    Notes

    max-limit must be equal or greater than limit-at.

    Simple queues are applied before queue trees.

    Queue rules are processed in the order they appear in the list. If some packet matches the queue rule, then the queuing mechanism specified in that rule is applied to it, and no more rules are processed for that packet.

    Example

    To add a simple queue that will limit traffic destined to 192.168.0.0/24 network on ether1 interface to 128000 bits per second: 
    [admin@MikroTik] queue simple> add dst-address=192.168.0.0/24 interface=ether1\
\... max-limit=128000
[admin@MikroTik] queue simple> print
Flags: X - disabled, I - invalid, D - dynamic
  0   name="queue1" src-address=0.0.0.0/0 dst-address=192.168.0.0/24
      interface=ether1 limit-at=0 queue=default priority=8 max-limit=128000

[admin@MikroTik] queue simple> print

    Configuring Queue Trees

    Submenu level : /queue tree

    Description

    The queue trees should be used when you want to use sophisticated data rate allocation based on protocols, ports, groups of IP addresses, etc.

    Property Description

    name (name; default: queue1) - descriptive name for the queue
    parent (name) - name of the parent queue. The top-level parents are the available interfaces (actually, main HTB). Lower level parents can be other queues. Dynamic queues (created with the simple queue tool) cannot be used as parents
  • global-in - match all incomming traffic
  • global-out - match all outgoing traffic
    flow (name; default: "") - flow mark of the packets to be queued. Flow marks can be assigned to the packets under /ip firewall mangle when the packets enter the router through the incoming interface
    limit-at (integer; default: 0) - maximum stream data rate (bits/s)
  • 0 - no limit
    queue (name; default: default) - queue type
    priority (1...8; default: 8) - flow priority, 1 is the highest
    max-limit (integer; default: 0) - maximum stream data rate (bits/s)
  • 0 - no limit

    Notes

    max-limit must be equal or greater than limit-at.

    To apply queues on flows, the mangle feature should be used first to mark incomming packets.

    If you have added a simple queue, it is listed as dynamic one in this list:

    Simple queues are applied before queue trees. 

    [admin@MikroTik] queue simple> print
Flags: X - disabled, I - invalid, D - dynamic                                  
  0   name="simple queue" src-address=0.0.0.0/0 dst-address=192.168.0.0/24     
      interface=ether1 limit-at=0 queue=default priority=8 max-limit=128000    
                                                                               

[admin@MikroTik] queue simple> .. tree
[admin@MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
  0  D name="simple queue" parent=ether1 flow="" limit-at=0 queue=default
       priority=8 max-limit=128000


[admin@MikroTik] queue tree>

    Example

    To mark all the thaffic going from web-servers (TCP port 80) with abc-http mark: 
    [admin@MikroTik] ip firewall mangle> add action=passthrough mark-flow=abc-http \
\... protocol=tcp src-port=80
[admin@MikroTik] ip firewall mangle> print
Flags: X - disabled, I - invalid
  0   src-address=0.0.0.0/0:80 in-interface=all dst-address=0.0.0.0/0:0-65535
      protocol=tcp tcp-options=any icmp-options=any:any flow=""
      src-mac-address=00:00:00:00:00:00 limit-count=0 limit-burst=0
      limit-time=0s action=passthrough mark-flow=abc-http tcp-mss=dont-change

[admin@MikroTik] ip firewall mangle>
    You can add queue using the /queue tree add command: 

    [admin@MikroTik] queue tree> add name=HTTP parent=ether1 flow=abc-http \
max-limit=128000
[admin@MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic                               
  0  D name="simple queue" parent=ether1 flow="" limit-at=0 queue=default
       priority=8 max-limit=128000

  1    name="HTTP" parent=ether1 flow="abc-http" limit-at=0 queue=default
       priority=8 max-limit=128000


[admin@MikroTik] queue tree>

    Troubleshooting

    Queue Applications

    One of the ways to avoid network traffic ‘jams’ is usage of traffic shaping in large networks. Traffic shaping and data rate allocation is implemented in the MikroTik RouterOS as queuing mechanism. Thus, the network administrator is able to allocate a definite portion of the total data rate and grant it to a particular network segment or interface. Also the data rate of particular nodes can be limited by using this mechanism.

    Example of Emulating a 128k/64k Line

    Assume we want to emulate a 128k download and 64k upload line connecting IP network 192.168.0.0/24. The network is served through the Local interface of customer's router. The basic network setup is in the following diagram:

    128/64k Line

    The IP addresses and routes of the MikroTik router are as follows:

    [admin@MikroTik] > ip address print
Flags: X - disabled, I - invalid, D - dynamic
  #   ADDRESS            NETWORK         BROADCAST       INTERFACE
  0   10.0.0.217/24      10.0.0.217      10.0.0.255      Public
  1   192.168.0.254/24   192.168.0.0     192.168.0.255   Local
[admin@MikroTik] > ip route print
Flags: X - disabled, I - invalid, D - dynamic, J - rejected,
C - connect, S - static, R - rip, O - ospf, B - bgp
    #    DST-ADDRESS        G GATEWAY         DISTANCE INTERFACE
    0  S 0.0.0.0/0          r 10.0.0.1        1        Public
    1 DC 192.168.0.0/24     r 0.0.0.0         0        Local
    2 DC 10.0.0.0/24        r 0.0.0.0         0        Public
[admin@MikroTik] >

    Assume you want to limit the data rate to 128kbps on downloads and 64kbps on uploads for all hosts on the LAN. Data rate limitation is done by applying queues for outgoing interfaces regarding the traffic flow. It is enough to add two queues at the MikroTik router: 

    [admin@MikroTik] queue simple> add name=Down interface Local max-limit 128000
[admin@MikroTik] queue simple> add name=UP interface Public max-limit 64000
[admin@MikroTik] queue simple> print
Flags: X - disabled, I - invalid, D - dynamic
  0   name="Down" src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Local
      limit-at=0 queue=default priority=8 max-limit=128000

  1   name="UP" src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Public
      limit-at=0 queue=default priority=8 max-limit=64000

[admin@MikroTik] queue simple> .. tree print
Flags: X - disabled, I - invalid, D - dynamic
  0  D name="Down" parent=Local flow="" limit-at=0 queue=default priority=8
        max-limit=128000

  1  D name="UP" parent=Public flow="" limit-at=0 queue=default priority=8
        max-limit=64000


[admin@MikroTik] queue simple>

    Leave all other parameters as set by default. The limit is approximately 128kbps going to the LAN and 64kbps leaving the client's LAN. Please note, that the queues have been added for the outgoing interfaces regarding the traffic flow.

    To monitor the traffic flow through the interface while doing file transfer, use the /interface monitor-traffic command: 

    [admin@MikroTik] interface> monitor-traffic Public once
    received-packets-per-second: 9
       received-bits-per-second: 4.32kbps
        sent-packets-per-second: 6
           sent-bits-per-second: 65.58kbps

[admin@MikroTik] interface> monitor-traffic Public once
    received-packets-per-second: 7
       received-bits-per-second: 3.36kbps
        sent-packets-per-second: 10
           sent-bits-per-second: 65.15kbps

[admin@MikroTik] interface> monitor-traffic Public once
    received-packets-per-second: 11
       received-bits-per-second: 5.66kbps
        sent-packets-per-second: 7
           sent-bits-per-second: 52.70kbps

[admin@MikroTik] interface>

    If you want to exclude the server from being limited, add two queues for it with max-limit=0 (no limit) and move them to the top: 

    [admin@MikroTik] queue simple> add name=Serv_D interface=Local \
\... dst-address=192.168.0.17/32 max-limit=0
[admin@MikroTik] queue simple> add name=Serv_U interface Public \
\... src-address=192.168.0.17/32 max-limit=0
[admin@MikroTik] queue simple> print
Flags: X - disabled, I - invalid, D - dynamic
  0   name="Down" src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Local
      limit-at=0 queue=default priority=8 max-limit=128000

  1   name="UP" src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Public
      limit-at=0 queue=default priority=8  max-limit=64000

  2   name="Serv_D" src-address=0.0.0.0/0 dst-address=192.168.0.17/32
      interface=Local limit-at=0 queue=default priority=8 max-limit=0

  3   name="Serv_U" src-address=192.168.0.17/32 dst-address=0.0.0.0/0
      interface=Public limit-at=0 queue=default priority=8 max-limit=0

[admin@MikroTik] queue simple> move 2 0
[admin@MikroTik] queue simple> move 3 1
[admin@MikroTik] queue simple> print
Flags: X - disabled, I - invalid, D - dynamic
  0   name="Serv_D" src-address=0.0.0.0/0 dst-address=192.168.0.17/32
      interface=Local limit-at=0 queue=default priority=8 max-limit=0

  1   name="Serv_U" src-address=192.168.0.17/32 dst-address=0.0.0.0/0
      interface=Public limit-at=0 queue=default priority=8 max-limit=0

  2   name="Down" src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Local
      limit-at=0 queue=default priority=8 max-limit=128000

  3   name="UP" src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Public
      limit-at=0 queue=default priority=8 max-limit=64000

[admin@MikroTik] queue simple>

    Example of Using Masquerading

    If masquerading is used for the local address space 192.168.0.0/24 of the client computers in the previous example setup, then the outgoing traffic has masqueraded source address 10.0.0.217, i.e., the outgoing packets have external address of the router as the source.

    If you use simple queues, as in the previous example, the queuing rule for incoming traffic should match the customer's local addresses, whereas the rule for outgoing traffic should match the router's external address as the source address. The previous example would work fine, but you cannot exclude the server from being limited.

    To apply specific queuing for the server, use /ip firewall mangle to mark the packets originated from the server: 

    [admin@MikroTik] ip firewall mangle> add src-address=192.168.0.17/32 \
\... action=passthrough mark-flow=Serv_Up
[admin@MikroTik] ip firewall mangle> add in-interface=Local action=passthrough \
\... mark-flow=Local-all
[admin@MikroTik] ip firewall mangle> print
Flags: X - disabled, I - invalid
  0   src-address=192.168.0.17/32:0-65535 in-interface=all
      dst-address=0.0.0.0/0:0-65535 protocol=all tcp-options=any
      icmp-options=any:any src-mac-address=00:00:00:00:00:00 limit-count=0
      limit-burst=0 limit-time=0s action=passthrough mark-flow=Serv_Up
      tcp-mss=dont-change

  1   src-address=0.0.0.0/0:0-65535 in-interface=Local
      dst-address=0.0.0.0/0:0-65535 protocol=all tcp-options=any
      icmp-options=any:any src-mac-address=00:00:00:00:00:00 limit-count=0
      limit-burst=0 limit-time=0s action=passthrough mark-flow=Local-all
      tcp-mss=dont-change

[admin@MikroTik] ip firewall mangle>

    Add a queue to the queue tree, which uses the flow mark: 

    [admin@MikroTik] queue tree> add name=Server parent=Public flow=Serv_Up
[admin@MikroTik] queue tree> add name=Workst parent=Public flow=Local-all \
\... max-limit=64000
[admin@MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
  0    name=Server parent=Public flow=Serv_Up limit-at=0 queue=default
       priority=8 max-limit=0

  1    name=Workst parent=Public flow=Local-all limit-at=0 queue=default 
       priority=8 max-limit=128000

[admin@MikroTik] queue tree>

    Thus, we used queue trees for limiting the upload. Use the same simple queues as in the previous example for limiting the download.

    Example of Guaranteed Quality of Service

    This example shows how to limit data rate on a channel and guarantee minimum speed to the FTP server allowing other traffic to use the rest of the channel.

    Assume we want to emulate a 128k download and 64k upload line connecting IP network 192.168.0.0/24 as in the previous examples. But if these speeds are the best that you can get from your Internet connection, you may want to guarantee certain speeds to the 192.168.0.17 server so that your customers could download from and upload to this server with the speeds not dependent on the other traffic using the same channel (for example, we will guarantee this server the minimum data rate of 32k for each flow direction).

    First of all, you should limit the interface speed: 

    [admin@MikroTik] queue tree> add name=Up parent=Public max-limit=64000
[admin@MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
  0    name="Up" parent=Public flow="" limit-at=0 queue=default priority=8
       max-limit=64000

[admin@MikroTik] queue tree>
    Next, mark the traffic from the FTP server. We will mark only TCP ports 20-21 because these ports are used to send and receive FTP data and control messages. 
    [admin@MikroTik] ip firewall mangle> add src-address=192.168.0.17/32:20-21 \
\... protocol=tcp mark-flow=Server_Up in-interface=Local
[admin@MikroTik] ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
  0   src-address=192.168.0.17/32:20-21 in-interface=Local
      dst-address=0.0.0.0/0:0-65535 protocol=tcp tcp-options=any
      icmp-options=any:any flow="" src-mac-address=00:00:00:00:00:00
      limit-count=0 limit-burst=0 limit-time=0s action=accept
      mark-flow=Server_Up tcp-mss=dont-change

[admin@MikroTik] ip firewall mangle>
    The second mangle rule will match the rest of the traffic from the network: 
    [admin@MikroTik] ip firewall mangle> add src-address=0.0.0.0/0 \
\... mark-flow=Local_Up in-interface=Local
[admin@MikroTik] ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
  0   src-address=192.168.0.17/32:20-21 in-interface=Local
      dst-address=0.0.0.0/0:0-65535 protocol=tcp tcp-options=any
      icmp-options=any:any flow="" src-mac-address=00:00:00:00:00:00
      limit-count=0 limit-burst=0 limit-time=0s action=accept
      mark-flow=Server_Up tcp-mss=dont-change

  1   src-address=0.0.0.0/0:0-65535 in-interface=Local
      dst-address=0.0.0.0/0:0-65535 protocol=tcp tcp-options=any
      icmp-options=any:any flow="" src-mac-address=00:00:00:00:00:00
      limit-count=0 limit-burst=0 limit-time=0s action=accept
      mark-flow=Local_Up tcp-mss=dont-change

[admin@MikroTik] ip firewall mangle>
    Finally shaping the traffic: 
    [admin@MikroTik] queue tree> add name=Server_Up parent=Up limit-at=32000 \
\... flow=Server_Up max-limit=64000 priority=7
[admin@MikroTik] queue tree> add name=Local_Up parent=Up limit-at=0 \
\... flow=Local_Up
[admin@MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
  0    name="Up" parent=Public flow="" limit-at=0 queue=default priority=8
       max-limit=64000

  1    name="Server_Up" parent=Up flow="Server_Up" limit-at=32000 queue=default
       priority=7 max-limit=64000

  2    name="Local_Up" parent=Up flow="Local_Up" limit-at=0 queue=default
       priority=8 max-limit=0

[admin@MikroTik] queue tree>
    Thus, we used queue trees for limiting the upload. The download speed can be limited the same way simply changing the interface names and matching the packets destinated to the server (use 'external' server address if you are using DST-NAT): 
    [admin@MikroTik] queue tree> add name=Down parent=Local max-limit=128000
[admin@MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
  0    name="Up" parent=Public flow="" limit-at=0 queue=default priority=8
       max-limit=64000

  1    name="Server_Up" parent=Up flow="Server_Up" limit-at=32000 queue=default
       priority=7 max-limit=64000

  2    name="Local_Up" parent=Up flow="Local_Up" limit-at=0 queue=default
       priority=8 max-limit=0

  3    name="Down" parent=Local flow="" limit-at=0 queue=default priority=8
       max-limit=128000

[admin@MikroTik] queue tree> /ip firewall mangle
[admin@MikroTik] ip firewall mangle> add dst-address=192.168.0.17/32:20-21 \
\... protocol=tcp mark-flow=Server_Down in-interface=Public
[admin@MikroTik] ip firewall mangle> add dst-address=0.0.0.0/0 \
\... mark-flow=Local_Down in-interface=Public
[admin@MikroTik] ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
  0   src-address=192.168.0.17/32:20-21 in-interface=Local
      dst-address=0.0.0.0/0:0-65535 protocol=tcp tcp-options=any
      icmp-options=any:any flow="" src-mac-address=00:00:00:00:00:00
      limit-count=0 limit-burst=0 limit-time=0s action=accept
      mark-flow=Server_Up tcp-mss=dont-change

  1   src-address=0.0.0.0/0:0-65535 in-interface=Local
      dst-address=0.0.0.0/0:0-65535 protocol=tcp tcp-options=any
      icmp-options=any:any flow="" src-mac-address=00:00:00:00:00:00
      limit-count=0 limit-burst=0 limit-time=0s action=accept
      mark-flow=Local_Up tcp-mss=dont-change

  2   src-address=0.0.0.0/0:0-65535 in-interface=Public
      dst-address=192.168.0.17/32:20-21 protocol=tcp tcp-options=any
      icmp-options=any:any flow="" src-mac-address=00:00:00:00:00:00
      limit-count=0 limit-burst=0 limit-time=0s action=accept
      mark-flow=Server_Down tcp-mss=dont-change

  3   src-address=0.0.0.0/0:0-65535 in-interface=Public
      dst-address=0.0.0.0/0:0-65535 protocol=tcp tcp-options=any
      icmp-options=any:any flow="" src-mac-address=00:00:00:00:00:00
      limit-count=0 limit-burst=0 limit-time=0s action=accept
      mark-flow=Local_Down tcp-mss=dont-change

[admin@MikroTik] ip firewall mangle> /queue tree
[admin@MikroTik] queue tree> add name=Server_Down parent=Down limit-at=32000 \
\... flow=Server_Down max-limit=128000 priority=7
[admin@MikroTik] queue tree> add name=Local_Down parent=Down limit-at=0 \
\... flow=Local_Down
[admin@MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
  0    name="Up" parent=Public flow="" limit-at=0 queue=default priority=8
       max-limit=64000

  1    name="Server_Up" parent=Up flow="Server_Up" limit-at=32000 queue=default
       priority=7 max-limit=64000

  2    name="Local_Up" parent=Up flow="Local_Up" limit-at=0 queue=default
       priority=8 max-limit=0

  3    name="Down" parent=Local flow="" limit-at=0 queue=default priority=8
       max-limit=128000

  4    name="Server_Down" parent=Down flow="Server_Down" limit-at=32000
       queue=default priority=7 max-limit=128000

  5    name="Local_Down" parent=Down flow="Local_Down" limit-at=0 queue=default
       priority=8 max-limit=0

[admin@MikroTik] queue tree>

    Additional Resources

  • Home of Hierarchical Token Bucket (HTB)
  • Paper on Random Early Detection (RED)
  • More complete information on Traffic Cotrol
    © Copyright 1999-2003, MikroTik