https://typesense.org logo
Join Slack
Powered by
Hi! I don't want that my users query typesense the...
# community-help
j
Hi! I don't want that my users query typesense themselves, Imagine attackers would ddose my typesense instances (or is there actually a protection?). So I would like that my user calls a google cloud function and the cloud function calls typesense. Is it OK like that? My app is worldwide, but my cloud functions, and my database, are both hosted in the US; so it would not make sense to use typesense clouds's SDN (spanning nodes across multiple regions)?
j
We have some amount of scraping protection - there's a parameter called "limit_hits" using which you can prevent more than X number of results from being returned. We've been talking about adding a simple IP-based Rate limiter to avoid DOS attacks, but that doesn't exist today. That said, one easy way to get DDOS protection currently is to setup Cloudflare DNS CNAMEs for each of the Typesense Cloud hostnames and proxy your requests via Cloudflare.
❤️ 1
So I would like that my user calls a google cloud function and the cloud function calls typesense. Is it OK like that?
This is also ok, but I don't think Google Cloud functions provide DDOS protection at least at the scale that Cloudflare offers. Also Cloudflare DNS is free for this use case 🙂
❤️ 1
j
@Jason Bosco THANKS THAT'S AWESOME!!! BTW so I could basically protect my Cloud Function endpoints too with Cloudflare DNS CNAMEs, right?
j
Yup!
That said, for Typesense specifically, you can go directly from your users' browser/mobile device to Cloudflare to Typesense Cloud.
❤️ 1
This way you can avoid the extra hop through Google Cloud functions, cold starts, etc and keep response times fast
❤️ 1
There are ways to generate user-specific API keys (called scoped API keys) that allow you to limit which users have access to which subset of their data in Typesense, so authorization is taken care off
j
So I won't even need to use cloud functions to access typesense from the cloud, brilliant!! - and this CNAME protection would be enough protection? (idk how CNAME works, it's not a simple domain forwarding or? (client -> xyz.com -> Cloudflare -> Typesense), because if it would be like the this attackers could just do client -> Typesense or not
j
Actually, I used the wrong word. It's not just a CNAME. Cloudflare actually proxies requests through their network. So you'd use Cloudflare as your domain's nameserver and then setup a sub-domain like typesense1.yourdomain.com in Cloudflare DNS and point that to xxx-1.a1.typesense.net, etc (one for each typesense node) So any requests made to typesense1.yourdomain.com actually get proxied through Cloudflare's network, and Cloudflare makes a call out to your Typesense Cloud nodes from their edge servers
❤️ 1
j
E.g I program an app like WhatsApp, can I create automatically millions of scooped API keys for auth, so that users will be able to ONLY search through their own messages? (i know it would be expensive af to do that, just askin cuz it's an easy example)
So attackers won't be able to find out the xxx-1.a1.typesense.net URL to ddos it? So I basically need to setup the URL like a uuid4 qpdjcjjdkeoe28384848ejrjdj-1.a1.typesense.net ?
j
E.g I program an app like WhatsApp, can I create automatically millions of scooped API keys for auth, so that users will be able to ONLY search through their own messages?
Yup!
(i know it would be expensive af to do that, just askin cuz it's an easy example)
It's actually not expensive to create scoped API keys. They're not actually created on the Typesense server-side. You can just generate one parent API key on the server-side and with that one key, you can create any number of scoped keys by cryptographically embedding filters inside the key, without touching the server. More details here (under scoped api key): https://typesense.org/docs/0.20.0/api/api-keys.html#generate-scoped-search-key
❤️ 1
So attackers won't be able to find out the xxx-1.a1.typesense.net URL to ddos it? So I basically need to setup the URL like a uuid4 qpdjcjjdkeoe28384848ejrjdj-1.a1.typesense.net
Cloudflare doesn't reveal the hostname(s) that it proxies to. So all your users will see is that requests are being made to typesense1.yourdomain.com That hostname points to a set of Cloudflare edge IPs. Behind the scenes, cloudflare will then proxy the call to the Typesense Cloud hostname. So your end users won't see the Typesense Cloud hostname anywhere for them to reach it
❤️ 1
Here's an example of how the cloudflare setup would look like (this is an example from an old Cloudflare DNS record I had): "Name" is subdomain and notice the IP address, and then notice that Proxy Status is "Proxied"
❤️ 1
Now if do a DNS lookup for tungsten.example.com (not the actual domain), here's what I get:
Copy code
$ dig <http://tungsten.example.com|tungsten.example.com>

; <<>> DiG 9.10.6 <<>> <http://tungsten.example.com|tungsten.example.com>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45761
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;tungsten.example.com.		IN	A

;; ANSWER SECTION:
<http://tungsten.example.com|tungsten.example.com>.	300	IN	A	104.26.10.144
<http://tungsten.example.com|tungsten.example.com>.	300	IN	A	172.67.68.165
<http://tungsten.example.com|tungsten.example.com>.	300	IN	A	104.26.11.144

;; Query time: 88 msec
;; SERVER: 192.168.7.1#53(192.168.7.1)
;; WHEN: Tue May 11 14:12:11 PDT 2021
;; MSG SIZE  rcvd: 97
Those 3 IPs are Cloudflare IPs and not the one I set in the Cloudflare DNS admin. That's because Cloudflare proxies the request through their network and the end user never sees the actual IP the request is being made to.
❤️ 1
j
So I basically do this on my cloud function: keyWithSearchPermissions = 'randomuuidv4keyfortheuser/orhisAUTHkey(google generates every 60 minutes a new one)'; thisIsTheKeyIGiveMyClient = client.keys().generateScopedSearchKey(keyWithSearchPermissions, {'filter_by': 'company_id:124', 'expires_at': 1611590465}) with thisIsTheKeyIGiveMyClient my Client can search with this key his company data, right?
j
with thisIsTheKeyIGiveMyClient my Client can search with this key his company data, right?
Correct.
keyWithSearchPermissions = 'randomuuidv4keyfortheuser/orhisAUTHkey(google generates every 60 minutes a new one)'
The parent search key is something you'd have to generate using the 
/keys
endpoint in Typesense. You don't have to necessarily rotate it every 60 minutes, you could if you want to of course. But typically you'd generate it once by hand say on your local machine, and then store that in your secrets vault (or however you distribute secrets to your cloud function) and then use that to generate scoped API keys on the fly for every user.
❤️ 1
j
Thanks a lot buddy! ❤️😭 from the picture: 198.245.61.162 is the private IP (like uuidv4wodjjxjskwiiwke-1.a1.typesense.net ) that I don't want anybode to see, correct?
j
That is exactly right!
❤️ 1
j
Ahh keyWithSearchPermission isn't a random uuidv4, it's the parent API master key (that can query eeeeverything) I have from typesense, correct?
j
That's correct
❤️ 1
j
damn i love you so much man I am so grateful that I meet you typesense guys 😭😭❤️❤️❤️😭😭
j
🤗 Thank you so much for all the support and feedback!
❤️ 1
j
j
Amazing, thank you!
🙌 1
r
just jumping in and saying Cloudflare is great, I use them for every single thing, personal or business related. With their free plan there's no reason to not use them. Unless you are concerned about privacy of course.
➕ 1
3 Views
Open in Slack
PreviousNext
Powered by