A server is offering public full-text search of our posts
Will Murphy
Public
Seen by 72
Came across this post announcing a public, full-text, cross-server search feature. I tried their search and found posts from our server. Full-text search is a tool used by targeted harassment campaigns, and it has been purposefully omitted from Mastodon. The post says we can request an opt-out via our admin account
Poll Created Tue 8 Nov 2022 3:46PM
CWG to contact @[email protected] via social coop admin account to request opt-out of search indexing Closed Wed 9 Nov 2022 5:49PM
Thanks for your participation all. The operator of the service has shut it down per this message on their homepage, https://fedsearch.io
>Due to extreme backlash from the Mastodon community we decided to end the project, it is obviously not wanted by server admins.
So this proposal is now moot.
If you're interesting in managing the indexing of your public posts, be sure to visit https://social.coop/settings/preferences/other and check out the "Opt-out of search engine indexing" option. This will tell well-behaved search engines not to collect your posts or offer them in search results. However, this does not prevent a bad actor from ignoring the setting and aggregating anyway. Take advantage of Mastodon's post visibility options to limit the distribution of messages that you do not want to be easily accessible to the public: https://docs.joinmastodon.org/user/posting/#privacy
Results
| Results | Option | % of points | Voters | |
|---|---|---|---|---|
|
|
Agree | 71.4% | 10 |
MM
S
|
| Abstain | 7.1% | 1 |
|
|
| Disagree | 21.4% | 3 |
G
|
|
| Undecided | 0% | 0 |
14 of 14 people have voted (100%)
Sam Whited
Tue 8 Nov 2022 3:50PM
Seems fine; posts and accounts can still be made private and discoverability while it can of course be a tool for harassment can also be a tool for finding interesting thoughts, conversations, and people to follow or engage with. Better discoverability seems good to me.
juniper cameryn
Tue 8 Nov 2022 3:53PM
Being able to find/be found by interests and topics without just hashtags sounds cool but I am more interested in minimizing the possibility of targeted harassment.
Eamon Caddigan
Tue 8 Nov 2022 4:28PM
The Fediverse has strong norms against full-text search. Regardless of how I feel about those norms (fwiw, I'm ambivalent), I think it's bad practice to break those norms and only offer the option to opt-out. The only right and decent way to introduce full-text search to this ecosystem is on an opt-in basis.
Giacomo
Tue 8 Nov 2022 4:46PM
I disagree with the idea, but I think the opt-out from that search engine should be an option for the individuals who prefer it. I would suggest to follow through with the proposal but in a different way
Will Murphy
Tue 8 Nov 2022 3:46PM
I need to rethink my vote after learning that their crawler attempts to honor the user-level search engine opt-out option found on https://social.coop/settings/preferences/other
(although at present it fails to honor it completely, I assume this was inadvertent)
Jonobie Ford
Tue 8 Nov 2022 5:13PM
I would change to disagree if it seemed like their user-level opt in/out was being respected (@Will Murphy 's comment that it exists but isn't working at all).
Moon Baron
Tue 8 Nov 2022 6:06PM
I concur with the points made by @sam and @giacomosansoni
Tom Resing
Tue 8 Nov 2022 7:43PM
They should honor the opt out found in user preferences and honor it completely (including boosted posts). I also posted in that thread that they should consider switching to opt-in for servers instead of opt-out.
shosha
Wed 9 Nov 2022 4:57PM
Agree with server-level opt-out in the absence of a fully functional individual-level opt-out. As others have said, also agree this kind of thing would be more appropriate as an opt-in (at whatever level). Also - seems like we kind of found out about this by accident? Is there anything we can do at the server-level to prevent this kind of thing? Or at least monitor for it?
Will Murphy Tue 8 Nov 2022 4:05PM
Also our server-level opt-out will not delete your data, we're each obligated to make a post after the opt-out in order to clear past data
Data removal
After opting-out you might want to remove your posts from the search engine. A simple post with the hashtag #RemoveMyContentFromSearchEngines will remove every data linked to your profile from the search engine. Make sure you're opted-out before or you will get indexed again.
Item removed
Will Murphy Tue 8 Nov 2022 4:25PM
This wasn't mentioned in the announcement post, but the site says there is a user-level opt-out via your profile's search engine discoverability setting. With this, there may not be any need for a server-level opt-out
Will Murphy Tue 8 Nov 2022 5:02PM
@Sam Whited the main concern is safety. Without this indexer, we had the ability to choose which content was discoverable and how it was discoverable by using hashtags. This search is now making posts discoverable by any portion of its content without the consent of the poster.
One of the main drivers of toxic interactions on twitter in my experience was full-text search. People looking for a fight would search on a controversial topic and then insert themselves into your day with negative posts.
Right now, alerts form social.coop make me happy. I don't want to go back to dreading notifications.
However, since making this post I've researched more and they do also offer user-level opt-out, so my stance on a server-level action has softened
Sam Whited Wed 9 Nov 2022 2:40AM
Isn't this the same as any other search engine on the internet though? We could already do a search (and not just by hash tags) on any given instance; I guess this makes it a little easier by aggregating a few instances, but a bad actor could just as easily use a search engine with a few keywords ("mastodon <controversial topic>"). And you can still choose what content is discoverable by using Mastodon's various privacy levels (this still can't see your private posts) so I don't really think this makes anything less safe or changes anything in any way.
Ana Ulin Wed 9 Nov 2022 6:05PM
The proposal is now moot, but I wanted to make the point that just because posts are already searchable in a regular search engine, that doesn't mean that adding a service like Fedisearch wouldn't "change anything in any way". On the contrary: adding fediverse services that shift the culture away from its current mutualistic, consent-based values is a big and consequential change.
Will Murphy Wed 9 Nov 2022 6:45PM
Personally, I've opted-out of search engine indexing for my Social Coop profile. The creator's announcement only mentioned server-level opt-outs as an option, so I brought this proposal so that myself and others who wish to be excluded from search results could maintain those wishes. After learning they also had attempted (incompletely) to honor the search engine opt our for individual profiles, I changed my vote
Will Murphy Wed 9 Nov 2022 6:47PM
Also, if someone wanted to offer fediverse search that was consensual by only indexing those who had opted-in (like #fedi22), I would be all for it
Sam Whited Wed 9 Nov 2022 11:57PM
Haven't you consented by making the post public? I'm all for getting consent, but I don't see how this changes the values in any way.
@[email protected] Wed 9 Nov 2022 4:27PM
@Will Murphy regardless of how this goes, can there be a toot about how to opt out of this? Today it's this server, but there will be others.
Darren Fri 11 Nov 2022 4:25PM
If Im not confused there wasnt much consideration for post privacy within the activitypub specifications.
As I understand it the limitations to searching only by hashtags was a Mastodon implementation and other activitypub server implementations dont necessarily have the same limitation, some certainly dont.
Theres been work on improving privacy, most notably via the AUTHORIZED_FETCH configuration implemented in Mastodon a few years ago.
https://docs.joinmastodon.org/admin/config/
Unfortunately it has significant impacts with compatibility with other activitypub servers so its off by default.
Over the years theres been a number of cases where its been noticed that activitypub posts have been scraped/indexed - theres the web search engines as has been mentioned, also academics doing research.
Theres also possibly unexpected ways in which posts, including Direct Messages can be handled across the federation
Generally I think its safest for people to consider all posts to be public until theres a robust privacy implementation thats widely adopted by activitypub servers
Although not hugely relevant to us, there is an alternative fediverse protocol called zot that has a major focus on post privacy which is used by Hubzilla, Zap, Streams etc.
Will Murphy · Tue 8 Nov 2022 4:03PM
Update: found the opt-out instructions and it's via email: Server opt-out
via https://fedsearch.io/privacy