[Adminsysters] [mur.at #9912] server disconnected

[info@ooooo.be via RT]

Hi
and we don't see this peak
:/ - i checked with nload and nethogs - but maybe we need btter 
minitoring.+ we are working on it *
xm


On 20/01/2022 22:57, Ralph Wozelka via RT wrote:
> and this screeny
>
> On 20.01.22 22:23, Ralph Wozelka wrote:
>> FYI, adele is maxing out its port limit hard!
>> Your services cannot be reached from the outside since 30 minutes.
>>
>> see scrshots attached
>>
>> cheers,
>> ralph
>>
>> On 20.01.22 21:38, Ralph Wozelka via RT wrote:
>>> Dear adminsysters,
>>>
>>> around 8pm CET our core switch went into saturation again.
>>>
>>> this time we put a rate-limit on right onto the network port restricting to
>>> 30Mbps in/out.
>>> this should do the trick finally.
>>> restricting traffic at our edge router was a few hops too late, obviously.
>>>
>>> see screenshot of traffic incoming from the core network at our edge route
>>> v767-fe0-r1ko.mur.at
>>>
>>> let us know how adele performs on your end! :-)
>>> cheers,
>>> ralph
>>>
>>>
>>>
>>> On Thu Jan 20 12:48:15 2022,info at ooooo.be  wrote:
>>>> FIY # we did a full backup again to check the restricted rate limit we
>>>> put, we expect the next backups to be incremental and much swifter.
>>>>
>>>> And we soon try to formulate a eeply.
>>>> Thanks already for the support.
>>>>
>>>> XmOn Jan 19, 2022 9:40 PM, Djamil Vardag via RT<amt at mur.at>  wrote:
>>>>> Hey you all
>>>>>> Hey ralph
>>>>>> Hey mur.at
>>>>>>
>>>>>> We are studying to put some bandwidth limit and monitor so we can
>>>>>> troubleshoot better the machine. But where to put this limit? in
>>>> our
>>>>>> side or in your gateway? And how much ? Like during the night we
>>>> can use
>>>>>> the entire bandwidth for the server upload and download..
>>>>>> and during the day (8:00-22:00) Adele can use... eg. 30MB/s for
>>>> upload
>>>>> You can use 20 Mbit up/down, that would be ok for now. We have to
>>>> discuss that
>>>>> internally.
>>>>>> and 10MB/s for download ? Or what do you suggest ? We need again
>>>> access
>>>>>> again - port 22 incoming is not enough to troubleshoot to check
>>>> our borg
>>>>>> backup script - for that we need for sure 22 outgoing port.
>>>>> Ok, for now we have tried to implement a iptables rate limit. You
>>>> should have
>>>>> full TCP access to adele again, UDP is still blocked.
>>>>> It would be ok for us if you use the available bandwidth as long as
>>>> our
>>>>> services are still working fluidly, but that isn't that easy to
>>>> implement.
>>>>>> It can also be our peertube instance. We disabled the webtorrent.
>>>> We
>>>>>> understand that peertube needs specific requirements -
>>>>>> https://docs.joinpeertube.org/install-any-os?id=installation
>>>>>> No adsl - you have a 100M/Bit symmetric fiber. Is that dsl? Do you
>>>>> This is a 100/100 MBit fiber, no DSL, the bottleneck is the fiber
>>>> and one
>>>>> 100MBit switch wich will get overwhelmed if there are to many
>>>> packets.
>>>>>> suggest that we can run our peertube instance on your
>>>> infrastructure?
>>>>>> Our users are very limited and moderated. Also we don't have a lot
>>>> of
>>>>>> material on it. It is rather an experimental instance.
>>>>> Ok, i guess this would't be a problem for our connection. Webtorrent
>>>> suggests
>>>>> that it uses less bandwidth on our end.
>>>>>> We do understand your decision but for the moment we our server
>>>> back
>>>>>> running.
>>>>> Sure, we can understand that and sorry that we had to be so drastic,
>>>> but we
>>>>> didn't have a choice as all of mur.at wasn't working well and we are
>>>> loosing
>>>>> our face in front of our members and financers.
>>>>>> We want to install a monitoring software like netdata -
>>>> promotheus.
>>>>> Seems like a good idea
>>>>>> Or do you have any suggestions to tackle the bandwidth problem? We
>>>> have
>>>>>> currently a funded project running until May so an online presence
>>>> +
>>>>>> visibility is importantand urgent- even while we are learning.
>>>>> The problem is that we don't have any QOS on our switches enabled
>>>> yet, to
>>>>> tackle rate limitting there. We need to learn and implement that
>>>> first.
>>>>> We tried establishing a 10Mbit link to your machine, but it didn't
>>>> like that.
>>>>> The reasoning being that you could still use the machine, and we
>>>> would have a
>>>>> pseudo rate limit.
>>>>> You mentioned a daily backup, maybe this is the culprit. As i said
>>>> your machine
>>>>> made traffic of 8TB in one week, that is just too much.
>>>>> There are some websites and services of us that are also funded and
>>>> we could
>>>>> get a problem with renewing these services, which could cost us
>>>> thousends in
>>>>> the future.
>>>>>
>>>>>> Can you also add this emailsystercloud at grrlz.net  to the mur.at
>>>>>> mailinglist
>>>>> we would suggest to putsystercloud at grrlz.net  as a forwarding
>>>> address of donna
>>>>> (donna at mur.at), who already is on all the lists.
>>>>>> xm


-------------- next part --------------
A non-text attachment was scrubbed...
Name: IMG_20220120_231845.jpg
Type: image/jpeg
Size: 1979018 bytes
Desc: not available
URL: <http://lists.genderchangers.org/pipermail/adminsysters/attachments/20220121/6361cbc4/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IMG_20220120_231908.jpg
Type: image/jpeg
Size: 2356105 bytes
Desc: not available
URL: <http://lists.genderchangers.org/pipermail/adminsysters/attachments/20220121/6361cbc4/attachment-0003.jpg>