rate-limiting for the channelserver

Discussion:

Ryan Kelly

2018-09-20 04:35:05 UTC

Hi All,

Over in github we've been discussing our options of rate-limiting pairing
channel creation attempts:

https://github.com/mozilla-services/channelserver/issues/21

One obvious approach would be to use the existing fxa-customs-server, and
just add some new action types like "createPairingChannel" and
"connetToPairingChannel" that the channelserver can send over for
checking. However, the fxa-customs-server is currently run as a private
"sidecar" service for fxa-auth-server, exposed only over a localhost
interface.

Does it make sense for us to try to extract fxa-customs-server into its own
standalone service that can be accessed by multiple consumers? Or is that
likely to be more work than just adding rate-limiting code directly into
the channelserver?

Cheers,

Ryan

Ryan Kelly

2018-09-20 04:51:53 UTC

Permalink

Post by Ryan Kelly
Hi All,
Over in github we've been discussing our options of rate-limiting pairing
https://github.com/mozilla-services/channelserver/issues/21
One obvious approach would be to use the existing fxa-customs-server, and
just add some new action types like "createPairingChannel" and
"connetToPairingChannel" that the channelserver can send over for
checking. However, the fxa-customs-server is currently run as a private
"sidecar" service for fxa-auth-server, exposed only over a localhost
interface.
Does it make sense for us to try to extract fxa-customs-server into its
own standalone service that can be accessed by multiple consumers? Or is
that likely to be more work than just adding rate-limiting code directly
into the channelserver?

Another option would be to try running a third-party ratelimiting daemon
that can be shared among different services, such as:

https://github.com/lyft/ratelimit
https://github.com/limitd/limitd

Which may be less work than adding custom rate-limiting code in
channelserver.

+ulfr for possible opinions from opsec team.

Cheers,

Ryan

Julien Vehent

2018-09-20 05:11:24 UTC

Permalink

+alm & g-k

Post by Ryan Kelly

Another option would be to try running a third-party ratelimiting daemon
https://github.com/lyft/ratelimit
https://github.com/limitd/limitd
Which may be less work than adding custom rate-limiting code in
channelserver.
+ulfr for possible opinions from opsec team.
Cheers,
Ryan

Phil Booth

2018-09-20 05:49:38 UTC

Permalink

Fwiw, I have two somewhat conflicting thoughts related to this:

1. The email service needs to grow rate-limiting soon and if there's a
standalone service available to use already, that will probably save me
some time.

2. Historically I've found the customs server code hard to grok. Part of me
dreads the prospect of working with it again.

Of course, extracting it into its own service might also involve improving
readability/maintainability, which would abate my dread. But if a 3rd-party
option can do the same job (I haven't dug into the ratelimit/limitd links
yet), I might be more inclined to take that option.

Post by Julien Vehent
+alm & g-k

Post by Ryan Kelly

Post by Ryan Kelly
Hi All,
Over in github we've been discussing our options of rate-limiting
https://github.com/mozilla-services/channelserver/issues/21
One obvious approach would be to use the existing fxa-customs-server,
and just add some new action types like "createPairingChannel" and
"connetToPairingChannel" that the channelserver can send over for
checking. However, the fxa-customs-server is currently run as a private
"sidecar" service for fxa-auth-server, exposed only over a localhost
interface.
Does it make sense for us to try to extract fxa-customs-server into its
own standalone service that can be accessed by multiple consumers? Or is
that likely to be more work than just adding rate-limiting code directly
into the channelserver?

_______________________________________________
Dev-fxacct mailing list
https://mail.mozilla.org/listinfo/dev-fxacct

Ryan Kelly

2018-09-21 22:05:16 UTC

Permalink

With apologies, I'm going to take this off the public list and redirect to
fxa-staff; there's nothing confidential here, but it feels like the sort of
discussion that could accidentally bring up some supposed-to-be-secret
details of our production security infrastructure.

Post by Phil Booth
1. The email service needs to grow rate-limiting soon and if there's a
standalone service available to use already, that will probably save me
some time.
2. Historically I've found the customs server code hard to grok. Part of
me dreads the prospect of working with it again.
Of course, extracting it into its own service might also involve improving
readability/maintainability, which would abate my dread. But if a 3rd-party
option can do the same job (I haven't dug into the ratelimit/limitd links
yet), I might be more inclined to take that option.

FxA does one interesting thing that I haven't seen in other third-party
tools, where we sometimes rate-limit by "breadth" instead of "depth". For
example, we allow you to call the /account/status endpoint for a single
email address as many times as you like. But we rate-limit the number of
different email addresses that you can call /account/status on in a certain
period of time.

Maintaining this capability may require us to continue to maintain our own
service rather than use something off-the-shelf.

Ryan

Post by Phil Booth

Post by Julien Vehent
+alm & g-k

Post by Ryan Kelly

_______________________________________________
Dev-fxacct mailing list
https://mail.mozilla.org/listinfo/dev-fxacct