Loomio
Tue 30 Apr 2013 11:42AM

Implement realtime chat, possibly using XMPP

D diasp_eu Public Seen by 575

follow up
https://github.com/diaspora/diaspora/issues/4076

I've searched all issues for "chat" and all which suggest implementing chat are closed.
There seems to be no explanation of why they are closed, so I am opening a new one.

This issue shall be about basic chat functionality:

seeing who is online
chat between two users
For multiuser chat, a separate issue is suitable.

I think it is critical for the success of Diaspora to have chat, so I am requesting that you add this to a milestone.Thank you.

F

Flaburgan Tue 30 Apr 2013 11:51AM

I think WebRTC is THE solution because really powerful (look at this demonstration).

As @dennisschubert said, the technology is not mature yet, but will be available for every Firefox and Chrome users this summer. I think that 80% of our users use one of these two browsers, as we do not currently support IE and Safari+Opera have small % of users on desktop, for me, a release with WebRTC during the summer is something acceptable. But we are here for debate :)

G

goob Tue 30 Apr 2013 1:09PM

I think that chat is one of the features which has been put on hold until the Diaspora API has been finalised and made stable, but one of our more technical members will be able to confirm whether or not that is the case.

But it may well be a good idea to discuss now what will be the best way to implement chat, and which protocol(s) will be most appropriate, now so that those decisions have already been made once it's time to start coding the chat facility.

S

Shmerl Tue 30 Apr 2013 2:36PM

WebRTC is an JavaScript API. It doesn't mandate any protocol which you can implement through it. For chat there is no need to build server functionality into Diaspora itself - it would be crude engineering and would be too restrictive. It's better to take ready and robust solutions (for example ejabberd), and to run the XMPP server alongside Diaspora pod (let's say on the same host). Diaspora just needs a UI to communicate through it. And if user wants to use standalone client - it will work as well, since XMPP is a standard protocol.

For media, just use XMPP/Jingle as usual. If you want to work on Jingle over WebRTC - be my guest, but I don't think there are any ready solutions, and this will require a lot of work potentially. Also WebRTC will require support from XMPP servers, since you can't simply route Jingle over WebRTC relying on clients alone.

M

mathis Tue 30 Apr 2013 7:05PM

ther was an old work of a diaspora video-chat made by vittorio cuculo: https://github.com/vcuculo/diaspora/wiki/Setup-chat-and-videochat i can contact him if he want to contibute again , it's an italian user.

R

Ruxton Wed 1 May 2013 2:26AM

while i'd love to see this i think it's something that should be focused on at a later date when other things like federation are all sorted. The back end to things like federation are likely to provide a better interface for something like this.

PG

Paul Greindl Wed 1 May 2013 7:02PM

@ruxton Yes of course, we can always delay everything to a later point of time, but then we'll never come anywhere...?

TS

Tom Scott Wed 1 May 2013 7:53PM

@shmerl first of all, you're wrong about your assumptions on WebRTC. it is in fact a peer-to-peer chat API, which is why it's so interesting to this project because it's decentralized, e.g., your chat messages never touch the actual web app server. in fact, the diaspora app is just to facilitate logging in and using the WebRTC service, if that's how you're instant messaging.

I like the idea of XMPP. I feel like it may be a neat experiment to see what other kind of data we can federate. The problem with XMPP is that it requires another server to be set up and maintained. It requires people who are already having trouble getting a Rails app installed to have to install/configure ejabberd or an equivalent server. Additionally, podmins hosting on Heroku are out of luck, because you can't run processes on any port other than :80, which is the one we need for the web app. So Heroku users would have to pay for another VPS just to house their XMPP server, or find space on an existing VPS for such a task. While I agree with you here that XMPP might be a more feature-rich and "trusted" solution to our problem, there's no question that using XMPP would increase the complexity and reduce understandability for many new developers. Which is why we've been so nervous about implementing it. The last thing Diaspora needs is its podmins feeling alienated.

I personally think that if our goal is chat, we should focus on some sort of decentralized solution, which is looking every day more like WebRTC. If our focus is on changing the way we federate data and getting chat "for free" along with it, then XMPP seems to be the way to go.

I would also wager that since we aren't responsible for any IM traffic over the Diaspora server(s), that WebRTC is "cheaper" solution than XMPP, simply because we don't need a central server to facilitate communication between two people.

S

Shmerl Wed 1 May 2013 8:12PM

@Tom Scott: No, I'm not wrong about WebRTC. It's not a chat API. It's a JavaScript framework for RTC media streams. See https://en.wikipedia.org/wiki/WebRTC
It doesn't dictate the signalling protocol (which can be Jingle or SIP for example). If you use XMPP, Jingle is the natural path for signaling protocol (which should be routed through WebRTC if you want a JavaScript browser solution).

Using XMPP should be a solid must, otherwise you'll be proliferating non interoperable IM networks. No thanks - there are tons of these out there which can't talk to each other. We don't need another one, just to put it into Diaspora. Better even, if someone is so desperate to have an IM, then take a normal standalone client, connect to any of many existing XMPP servers - and that's it. No reinventing the wheel needed.

I understand the issues of running an extra server. But it only makes sense to do so, since it clearly separates Diaspora from IM service. Yes, Heroku is not an option. But Heroku limitations and idiosyncrasies should not dictate Diaspora design and limit your choices. Just don't use Heroku of you need an accompanying IM service.

TS

Tom Scott Wed 1 May 2013 8:58PM

@shmerl "Just don't use Heroku"? There are a lot of pods already on Heroku (disclaimer: mine included). This is an unsustainable suggestion, and it's kind of a big "fuck you" to all of the podmins who are hosting free/personal pods on Heroku. I'm not going to put others through that just because I want to use Diaspora as my IM provider.

"WebRTC (Web Real-Time Communication) is an API definition being drafted by the World Wide Web Consortium (W3C) to enable browser to browser applications for voice calling, video chat and P2P file sharing without plugins."

To me, that indicates that WebRTC does not require a server. Perhaps it isn't necessarily a protocol for chat, but I'm pretty sure that WebRTC is a peer-to-peer technology which means its data is not routed through a central server. This is a big win for us, because we are always looking for better solutions to the centralization problem. It ensures that the only 2 people who see the data are the 2 people having the conversation.

I like that a lot for personal chat communications, as long as this discussion is focused on that particular problem. What I don't think we have in this discussion is a grip on what the problem is that we're trying to solve. Are we just trying to give our users a basic chat service? Or do we want to use the XMPP protocol for something a bit beyond chat services, like perhaps as a means to re-invent the Federation Protocol?

TS

Tom Scott Wed 1 May 2013 8:59PM

Another disclaimer: I can easily run an ejabberd on the VPS that I already own, so that particular problem isn't something I have to solve for my own use case, but I can see how it would be an issue for others...

G

goob Wed 1 May 2013 9:01PM

@paulgreindl the problem is if features are developed before federation has been resolved (and stripped into its own code layer) and before an API has been finalised, those features might suddenly become broken when the code base changes at a later date. This has already happened with a chat feature and many other Diaspora features about a year ago, which put development back a long way when many developers became disillusioned at this happening. I think it's important to make sure this doesn't happen again, and so to focus on making the underlying code base stable enough so that features then added won't get broken when other parts of the code base change.

S

Shmerl Wed 1 May 2013 9:12PM

@Tom Scott:

"Just don't use Heroku"? There are a lot
of pods already on Heroku. This is an
unsustainable suggestion

It means don't use Heroku for XMPP server. If you need one - find another hosting to dedicate to IM part of your pod. If you can't - don't use XMPP server. Heroku should not limit Diaspora design - no one forces anyone to use it and Diaspora should not mandate any limitations based on Heroku preference.

that indicates that WebRTC does not require
a server. Perhaps it isn't necessarily a protocol for chat

Exactly, it's not a protocol for chat, nether it is a signaling protocol which is needed for sensible P2P media communications (most commonly used signalling protocols are SIP and Jingle). It's up to developers do decide what to route through WebRTC. See http://www.webrtc.org/faq#TOC-Why-should-I-use-WebRTC-
As I said, it makes sense not to reinvent the wheel, and to use existing protocols. Otherwise you'll end up with another walled service which won't be interoperable with anything outside itself.

S

Shmerl Wed 1 May 2013 9:20PM

Again, this discussion has nothing to do with Diaspora federation. It's about chat (IM/media) functionality. Federating the IM part should not be restricted to federating D*. I.e. if you use XMPP for IM part, it's automatically federated to the whole XMPP network. It doesn't relate to where D* itself is federated (as a social network).

G

goob Wed 1 May 2013 9:33PM

Shmerl, if that's a response to what I said, my mentioning federation is because that is one of the biggest things to be solved, and until the federation code has been put into its own layer, any future changes to the federation code (of which there will by necessity by many, possibly almost totally changing the federation code) risk breaking features such as chat, which has already happened once before. Therefore it makes sense to either fix and make stable the fundamental stuff such as federation and the API, or to put them in their own layers so that future changes to them don't affect other things, before spending too much time creating 'window-dressing' features.

I think Diaspora would struggle to survive another catasrophe in which loads of features, carefully created by developers over much time, suddenly broke because of a necessary change in the fundamental network code. We'd do well to avoid this happening again.

S

Shmerl Wed 1 May 2013 10:12PM

@Goob: Sure, fixing D* federation should be a priority I think. But I was answering to @Tom Scott, who for some reason tried to tie together D* federation and the choice of XMPP for IM component. I was saying that these are separate subjects, and should not be put together in this thread (except as you said, putting D* federation as a first priority).

PG

Paul Greindl Thu 2 May 2013 5:09AM

@goob This is absolutely valid for federation dependent features but not for a XMPP based chat...

RF

Rasmus Fuhse Thu 2 May 2013 7:40AM

My opinion is that an xmpp-chat for diaspora-federation is only a matter of a user-flag that says "this user has the following xmpp-chat-account". The rest (having a web-client for the chat, having a jabberd-server on the pod or an external server/account, or using WebRTC) is not affecting the diaspora-federation in any way. So using XMPP seems quite nice to me and should be able to be done even today, when federation is not put into it's own layer yet.

F

Flaburgan Thu 2 May 2013 8:33AM

Okay so this debate is more or less a choice between XMPP and WebRTC. As a Mozilla contributor, I'll contact some WebRTC guys asking if they can come here because I think we do not have enough knowledge to take a decision.

Anyway, here are the arguments for the moment:

Maturity:
XMPP is far more mature and supported than WebRTC, no doubt. WebRTC works on only few browsers for the moment, and this is a problem. Moreover, specifications can change because the techno was not released in stable for the moment.

Interoperability
XMPP is used by a lot of network (Jabber, GTalk, Facebook...). Unfortunately, Facebook is completely closed and Google started to close gtalk too (you cannot add a gtalk contact from another xmpp account for few weeks now). Anyway, there are a lot a websites / services (jabber, movim, libertree, etc) which used XMPP, far more than WebRTC and there are desktop clients for XMPP too. So for the moment, XMPP wins. In the other hand, it's possible that WebRTC becomes the replacement of XMPP in the future...

Complexity
About development, we do not know enough WebRTC to compare to XMPP implementation.
About pod administration, has said, XMPP is more complex than WebRTC. It needs another server, more sysadmin stuff, more server bandwidth / power, special configuration (ports other than 80)... and is not completely decentralized as it needs to use a server.
Remember, in a perfect Diaspora world, everybody has his own pod under his bed, in a raspberry. I see the server part becomes more and more complex with the time, and I don't like that. WebRTC totally solves this problem by processing between browsers directly.

Features
Here is the killer point in favor of WebRTC. Chatting, audio, video, file sharing, conf call... (remember, this is "the" killer feature of g+, officially...). This is AWESOME.

Moreover, we are in a moment where companies are trying to put social in them. If we add such features, Diaspora can be interesting for them. And companies can put money / developers in the projects which interested them. This is a really important point, we can maybe find a business model here (and we need one!!).

JR

Jason Robinson Thu 2 May 2013 12:09PM

A big huge +1 for XMPP chat support.

In addition to an actual chat UI, we only need to add a field in the user settings about the users XMPP handle. If the pod provides one, the user can opt to use that one. If the user does not like it or has an existing one, they will probably use that one.

I would probably use my Facebook XMPP handle since that is where I do most of my chatting with my friends.

We have a gazillion of things to do - let's not start building a platform for chatting as we can leverage on XMPP for that and just offer a simple client in the UI AND the possibility for podmins to run a server tied to their pod. If they want, Heroku podmins don't need to do if they don't want. Or they can run one on their VPS.

JR

Jason Robinson Thu 2 May 2013 12:10PM

Personally I don't need an additional chat handle - allowing Diaspora* users to contact me easily via my existing XMPP handles would be cool though.

F

Flaburgan Thu 2 May 2013 12:33PM

I would probably use my Facebook XMPP handle since that is where I do most of my chatting with my friends.

But by doing that, you will not be able to talk to me who have a Jabber account, aren't you?

Btw, integrate in Diaspora an existing account could be a good idea, but how do you add contacts? When you start sharing with someone, it automatically send a request to his xmpp account? And how the other user answer? The asymmetric relationship of Diaspora is really different of the symmetric relationship in xmpp..

JR

Jason Robinson Thu 2 May 2013 12:59PM

OK maybe Facebook XMPP based chat engine was a bad example. But anyway, something existent which is widely used, like @shmerl has been saying

S

Shmerl Thu 2 May 2013 4:30PM

@Flaburgan:

Okay so this debate is more or less a choice between
XMPP and WebRTC.

I think you are missing a point. XMPP and WebRTC are incomparable - they are different level frameworks. XMPP is a complete IM protocol, with signalling extensions like Jingle, which use P2P RTC for media chat (video/audio).

WebRTC is not a protocol, it's a low level JavaScript API which allows using RTC from the browser. I.e. it doesn't dictate any protocol whatsoever.

Therefore it's not a question of XMPP vs WebRTC. It's a question of what protocol do you want to use for chat? And if you use WebRTC for conferencing part of that chat (which makes sense for browser based client), what protocol do you want to implement through it? Remember again,WebRTC is not a protocol. See it as a tool, through which you implement your media protocol in JavaScript.

The way I see it - full blown and feature rich chat should look like this:

  1. IM component should use XMPP protocol (for example using strophe.js preferably with WebSockets, instead of BOSH).
  2. Conferencing component (video/audio chat) should use XMPP/Jingle signalling protocol, utilizing WebRTC framework to route it from the browser.

I hope this clarifies things a bit.

S

Shmerl Thu 2 May 2013 4:35PM

Anyway, my advice to anyone who proposes to use WebRTC in any way - read first about what it is, and what it isn't.

S

Shmerl Thu 2 May 2013 4:47PM

@Jason Robinson:

I would probably use my Facebook XMPP
handle since that is where I do most of my chatting
with my friends.

I agree that users should be able to set their XMPP handle, which can be from another server, not necessarily from the server shipped with the pod. However it has to be a federated server naturally. Facebook XMPP server is not federated, so we can forget about it right away. The fact that it's XMPP becomes irrelevant, since it's cut off from the XMPP network.

Google Talk is federated. Their invite blocking problems were temporary and they already resolved that issue.

C

Christophe Thu 2 May 2013 7:51PM

I'm running a XMPP server and a Diaspora pod side by side at wk3.org, so the users already have matching handles.

Since nobody came up with SIP yet, I'd like to suggest that. There's already a HTML5 client: sipML5 And we totally need to consider CU-RTC-WEB too.

S

Shmerl Thu 2 May 2013 11:39PM

SIP is fine for media, except that it doesn't integrate with XMPP, instead it integrates with its own IM protocol - SIMPLE. However Jingle is a natural extension of XMPP, and if you are already using XMPP for IM/chat, it makes sense to use Jingle rather than SIP.

RF

Rasmus Fuhse Fri 3 May 2013 5:14AM

Like Shmerl said, WebRTC is just a channel that can be used for any kind of content. Yet it will be easiest to implement video-chat and phone-calls since it has the best developed libraries.
My question would be (and I'm not sure what the answer is): is it possible to fully mimic an XMPP with webRTC? There are some asynchronous components in XMPP that would need to be done by the Diaspora-server like "does the user exists", "give me metadata of the user" and "send a friendship request", probably more. I am quite sure that this can't be done entirely just by using WebRTC.

But it would be definitely a good architecture, if the user can opt if he/she wants to use a) an existing XMPP-account, b) a provided XMPP-account by the pod (like Christophe's solution) c) a WebRTC-client for XMPP or d) no XMPP account at all.

If we want to focus not only on chatting, but also on video-calls and this stuff, we should consider this probably now, because this can be hardly be done by XMPP only. I have no idea what would be the way for that.

JR

Jason Robinson Fri 3 May 2013 7:19AM

I see this discussion finally moving in a good direction :)

S

Shmerl Fri 3 May 2013 7:59AM

@Rasmus Fuhse: For text messages you don't need WebRTC. It's a clear overkill. JavaScript offers you WebSockets already. WebRTC is targeted for media streams (i.e. audio/video), that's why it's dealing with codecs and the like. WebSocktes is a generic API for duplex communication through JavaScript. It fits perfectly for text part of XMPP. Some support for that is required on the backed (before WebSockets, the only method from the browser was BOSH). Now servers like ejabberd are catching up with WebSockets support. For media, WebRTC is an obvious API, but as I said - the protocol still needs to be defined, and it makes sense to use Jingle there:

#1. Text XMPP can work like this:
JS client (browser 1) <-> XMPP over WebSockets <-> XMPP Server <-> XMPP over WebSockets <-> JS client (browser 2)
#2. Media XMPP (Jingle) can work like this:
JS client (browser 1) <-> Jingle over WebRTC <-> JS client (browser 2).

Except that in #2 you still need some support of such routing from the servers, since there are security limitations. See some discussion in the comments here:

https://hacks.mozilla.org/2012/09/full-webrtc-support-is-soon-coming-to-a-web-browser-near-you/

S

Shmerl Fri 3 May 2013 8:02AM

The benefit of using XMPP and Jingle is that instead of JS client you can put any client (Pidgin, Jitsi etc.), and it would still work, since it will talk a standard protocol.

TS

Tom Scott Fri 3 May 2013 2:37PM

@shmerl If we are speaking of adding XMPP as an IM service only, then we may have a reasoning behind this. We can use XMPP for a lot more than just IM, if we were so inclined. I believe both Facebook and Twitter are/were using XMPP for realtime communication between their client side and server side code.

If we do this, I'd like to give the user the freedom to run XMPP or not. If they don't want it, they can disable its use. If they do want it, it is their responsibility to provide a server and XMPP protocol support on the backend. Diaspora is essentially just providing a nice web-based client to use with an XMPP server.

For a first version, I feel like it'd be easier to just focus on text chat, and then move on to voice/video implementation later. Are there free software implementations of this on the client side I can take a look at? Also, what is everyone's consensus on using a Flash component to handle video chat for regression into older browser clients?

S

Shmerl Fri 3 May 2013 3:17PM

@Tom Scott: Yes, we can use XMPP for Diaspora's own protocol, but as I said, this is out of scope for this particular discussion.

Back to IM: XMPP surely can be optional. Podmin can simply not associate the pod with XMPP server (and not install that server). There is no need to make it mandatory.

Regarding clients - Text/IM libraries are already around. Example:
http://strophe.im/strophejs/
https://gist.github.com/processone/739147

Audio/video conferencing and Jignle support through JavaScript is still catching up and you'll have to do the research. But that can be put to phase 2, with text IM being the first phase.

C

Christophe Sun 5 May 2013 2:13PM

Who is the person requested to implement this feature?

JR

Jason Robinson Sun 5 May 2013 6:32PM

@christophe - chat has been one of the most wanted features since the beginning, requested by many people

F

Flaburgan Mon 6 May 2013 7:58AM

@tomscott

Are there free software implementations of this on the client side I can take a look at?

Look at Jappix Mini ;)

what is everyone's consensus on using a Flash component to handle video chat for regression into older browser clients?

I'll block anything using Flash. This technology does not exist anymore :p

TS

Tom Scott Mon 6 May 2013 12:21PM

@flaburgan uhh, i think you misunderstood my question. but it was a stupid question anyway, because i'm pretty sure we can get websockets to work all the way down to IE 6 without using flash.

@shmerl i do know about Strophe. should be a decent choice for communicating with the XMPP server. let's focus on text chat right now. would Jappix Mini or the Facebook IM design work for our purposes?

TS

Tom Scott Mon 6 May 2013 12:21PM

and by "Jappix Mini" i mean the way it looks, not the product itself. we can't really use Jappix as a product because we don't control its code.

DU

[deactivated account] Tue 7 May 2013 7:16AM

How would you link Diaspora accounts with XMPP accounts?

Do you intend to allow users to provide an existing XMPP account or should they register a new account (on the connected XMPP server) through Diaspora? (see XEP 0077)

TS

Tom Scott Tue 7 May 2013 9:16PM

@rekado when i was playing around with ejabberd, i found the easiest solution would be to simply create an XMPP user for each diaspora user on the pod. and when new users are added/removed, we create/destroy their corresponding jabber user. seems pretty easy since both of them are using a relational DB.

DU

[deactivated account] Wed 8 May 2013 4:06AM

@Tom: I would advise against modifying the DB directly as you'd couple Diaspora too tightly with a particular XMPP server implementation (and maybe even version). XEP 0077 is the standard way to register accounts through XMPP. (My apologies if this is what you meant.)

CG

Christian Giménez Thu 9 May 2013 3:18AM

Hi to all!
This is my first time I use loomio so please have patience!

I don't know how much voice and votation I have, but I give my personal opinion about this:

I met lot of people that wants to have a chat feature, not a voice call neither video calls. Also I met people that just said "ah! that doesn't have a chat: it's boring!" so I cannot convince him to use D*. In other words: people got acustom to chatting instead of talking.

One problem about voice and video is not the implementation, is the Internet "speed" and bandwith. If you're going to add voice or video, please be sure to make it clearly optional.
If you ask my opinion about this, we should concentrate on chat. Video and calls are too much and has its own consequences(bandwith, services, privacy, etc.). At least, we should have a chat!

About XMPP, it do support creating counts through its protocol(I remember Pidgin asking if I want to create the account or use an already created). Then, the problem is if ejabberd or jabber.org will may be overpopulated and if they can stand so much people registered.
This makes clear why should be a separate service and not hosted in Diaspora servers, maybe some pods cannot stand such load. Diaspora can give an easy and beautiful
web interface, but the rest is up to jabber.org(and we support Jabber!).

I haven't take a look into WebRTC, but for now I vote in favor of XMPP: is mature enough for give more robustness to D*.
Cheers!

JR

Jason Robinson Thu 9 May 2013 6:13AM

@christiangimenez - everybody has a voice on Diaspora* :) Welcome!

TS

Tom Scott Thu 9 May 2013 3:58PM

@rekado at this point, i think just supporting a single XMPP server would be sufficient for our needs, but regardless if we can use a standard to make sure that our code is "future-proofed" for any other standards-compliant XMPP server that gets invented in the future, that seems like the "right way" to go.

TS

Tom Scott Thu 9 May 2013 4:00PM

@rekado to answer your question, i think XEP-0077 (http://xmpp.org/extensions/xep-0077.html) is a fine spec to go by when registering accounts on the XMPP server. there is an XMPP lib for Ruby, so we can send messages to the configured server like when a user is registered through the web interface.

DU

[deactivated account] Fri 10 May 2013 1:08AM

Actually, there's more than one XMPP lib for Ruby. I recommend using blather (rather than XMPP4R), because of its cleaner design thanks to running inside an EM loop.

FS

Florian Staudacher Fri 10 May 2013 1:58PM

so, this is the direction I see this discussion going:

D* associates a Jabber ID with a D* user. That could be a user-specified, existing JID - or - D* offers to create a Jabber account on a user-specified server via XMPP protocol (could have a nice auto-complete for a few known servers).
(the password would need to be stored in the DB unhashed, but we could at least encrypt it with the users private key)

The UI offers a 'chat' feature, that's more or less simply a Jabber client in the browser, connected to whichever account is specified in the D* users profile.
The exact implementation details are still to be decided...

SVB

Steffen van Bergerem Fri 10 May 2013 5:07PM

@florianstaudacher I just signed up because I wanted to write exactly what you already said ;-)

Just adding this: When user A starts sharing with user B the JID of A could be send to B. When B also starts sharing with A (so both know each others JID) we could add the JIDs to the jabber contactlist

SM

Seth Martin Fri 10 May 2013 6:55PM

It appears that many existing XMPP users will have to either get new accounts or install BOSH on their server to use such a feature.

CG

Christian Giménez Fri 10 May 2013 8:38PM

@steffenvanbergerem do you mean that user A adds B or just display it?
If adding, D* should ask in which group to add B.
This is useful when we're talking that D* server forward messages to Jabber servers.

@sethmartin what @florianstaudacher mean is that D* will only associate an XMPP account to the user, and then will only act as a client(maybe using Javascript or other).
If you have an existing account you associate it with your D* user.

I suppose that there's no need to alter the server functionality, the problem reduces to find a good free jabber client to embed into the webpage. Or, if I forgot something about securities issues, the server just act as a passarel, fowarding messages to the Jabber server.
Am I right? :-S

CG

Christian Giménez Fri 10 May 2013 8:47PM

I found lots of free(as in freedom) Jabber clients and libraries written in JavaScript: JSJaC, Candy. Take a look at the Jappix Mini feature looks interesting...

Hope this helps a bit more...

JH

Jonne Haß Fri 10 May 2013 9:59PM

@christiangimenez all of these require BOSH because that's the way to write XMPP clients for the browser....

CG

Christian Giménez Fri 10 May 2013 10:37PM

You're correct, it use BOSH. I learn something new! :-P

DU

[deactivated account] Sat 11 May 2013 12:12AM

If you were fine with the idea that users had to create a new XMPP account through Diaspora, then you would not need to use BOSH and the JS side of things could be rather simple (= no need for a full XMPP client in JS). Instead of distributing a full client to the users, you could push messages through a websocket in either direction; the websocket server process then takes over the role of an XMPP client.

The benefit here is: no need for BOSH, smaller JS footprint, easier to customise the chat interface, you can use Ruby + blather to interface with the XMPP server in an event loop rather than using the chopped up XMPP-via-HTTP thing that is BOSH.

FS

Florian Staudacher Sat 11 May 2013 10:56AM

yeah sure, that way we could achieve much tighter integration with our application, but it would also increase the work required to do this feature.

Also, I'm no expert on Rails but the googling I have done lately indicated that Rails is not particularly good at doing anything else apart from HTTP request-response cycles. Websockets and all the other nice tricks require a separate event loop that's outside the Rails app, meaning additional ports would be needed to be opened for D*. (Please correct me, if I am wrong.)
So what I'm saying: a standalone chat server in ruby appears to be no problem, but integrating one in D* - even if it's just to relay XMPP messages to the client browsers - is a completely different story.

DU

[deactivated account] Sun 12 May 2013 12:12AM

Websockets and all the other nice tricks require a separate event loop that's outside the Rails app, meaning additional ports would be needed to be opened for D*

To support websockets you only need to run a websocket server. This is a simple, separate process (e.g. written in Ruby) that dispatches on messages that come in via websocket. This process can (and should) operate independently from the Rails application. Thanks to library support for websockets, it's very simple to make this work. The separate process could be spawned during initialisation of the Rails app, so it doesn't even need to be started separately.

One additional port is needed to be open, but I don't consider this a problem --- it's certainly much less of a problem than installing and configuring an XMPP server.

JH

Poll Created Sun 12 May 2013 7:44AM

Find a solution for chat that works on (free) Heroku Closed Wed 22 May 2013 7:14AM

Outcome
by Jonne Haß Tue 25 Apr 2017 5:54AM

We don't need a solution that works with a free Heroku account alone but can mandate an external service. But it needs to able to communicate with pods installed on Heroku.

Yes: Yes, free should work too.
Abstain: I tend to want it but free is not mandatory.
No: Lets screw our Heroku users.

Given that our biggest pod runs on Heroku and that the discussion all goes into the direction of separate ports/servers/processes lets decide this once and for all to ease the discussion.

Results

Results Option % of points Voters
Agree 34.8% 8 ST FS JH SH DM D R CG
Abstain 43.5% 10 TS JR EG G DS M D MS SVB S
Disagree 21.7% 5 RF T S DU SM
Block 0.0% 0  
Undecided 0% 250 BK MS AA S CB HF BO DM GC JH F RF M G AX PC PP BB LP T

23 of 273 people have voted (8%)

ST

Sean Tilley
Agree
Sun 12 May 2013 7:49AM

This needs to happen. We cannot create a feature disparity between our pods just based on what host a podmin is on; that creates more problems than it solves. :/

S

Shmerl
Agree
Sun 12 May 2013 7:54AM

If such solution is possible with XMPP - why not. But it seems rather non trivial. Better evaluate if this effort is really required, or there are easier workarounds.

SVB

Steffen van Bergerem
Abstain
Sun 12 May 2013 9:20AM

I think we could let the podmin configure if he/she wants xmpp support. In that case heroku users will get a working installation of diaspora - just without chat.

SM

Seth Martin
Disagree
Sun 12 May 2013 7:37PM

I'm not interested in any other solution than XMPP because I want to take my chats with me when leaving the browser. I hardly consider it screwing Heroku users by requiring they use a third party XMPP server if they want to chat.

CG

Christian Giménez
Agree
Sun 12 May 2013 10:19PM

I want support for XMPP. If Heroku can give support for XMPP, for me is OK.

Nevertheless, I agree with Seth: We cannot trust or data on third parties people.
I turn into "yes" because Chat feature is really needed.

CG

Christian Giménez
Agree
Sun 12 May 2013 10:20PM

I want support for XMPP. If Heroku can give support for XMPP, for me is fine.

Nevertheless, I agree with Seth: We cannot trust or data on third parties people.
I turn into "yes" because Chat feature is really needed.

S

Shmerl
Disagree
Sun 12 May 2013 10:33PM

If such solution is possible with XMPP is possible on Heroku - why not. But if not, better evaluate if this effort is really required, or there are easier workarounds, like simply using an external XMPP server on the other hosting.

S

Shmerl
Disagree
Sun 12 May 2013 10:34PM

If such solution with XMPP is possible on Heroku - why not. But if not, better evaluate if this effort is really required, or there are easier workarounds, like simply using an external XMPP server on the other hosting.

DU

[deactivated account]
Abstain
Sun 12 May 2013 11:21PM

"Abstain" really means "abstain" here. If it was my project I wouldn't want to be limited by the feature set offered by one hosting provider. (Embedding a full XMPP client in Diaspora kind of defeats the purpose, in my opinion. Embedded < dedicated).

TS

Tom Scott
Abstain
Mon 13 May 2013 12:33AM

You should be able to run a pod, for free, on Heroku. Without having to set up ejabberd or some other XMPP server. If more advanced users want more advanced things, more power to them.

I don't use the CloudFront features, but I'm glad they exist.

JR

Jason Robinson
Abstain
Mon 13 May 2013 10:38AM

Heroku should not dictate our plan for chat. As long as we don't break existing functionality in Heroku pods.

RF

Rasmus Fuhse
Disagree
Mon 13 May 2013 12:31PM

Let there be XMPP and give the chat-module the possibility that singe diaspora-accounts might not be linked to an XMPP-account.

G

goob
Abstain
Mon 13 May 2013 9:17PM

One the one hand, good to find a solution which works with current pod hosts. On the other hand, might be best to find best chat solution for the Diaspora network as a whole, and podmins can then find a host which can provide this. I can't decide.

DS

Dennis Schubert
Abstain
Fri 17 May 2013 7:50AM

I am really into that "let everybody run an own pod on heroku for free"-thingy, but I doubt this is the way we should make our decisions.

Also note https://github.com/diaspora/diaspora/issues/4076?source=cc#issuecomment-17213642 (re: WebRTC)

DU

[deactivated account]
Disagree
Mon 20 May 2013 1:51AM

"screw Heroku users" <--- No. But if it was my project I wouldn't want to be limited by the feature set offered by one hosting provider.

EG

Erwan Guyader
Abstain
Mon 20 May 2013 5:43PM

Even if I like the free Heroku platform I'm not expecting big and complex applications to run on it.
And if hosting a separate server for the chat is a solution, I don't see any problem in not having it work on FREE Heroku.

S

Shmerl Sun 12 May 2013 7:52AM

BOSH is not the only way, and not a preferable one. Better one is Websockets.

As for Heroku - that will require to come up with XMPP, I'm not sure that Heroku is at all geared towards that. It seems to be targeted only to web applications, not to other type of servers. So pushing XMPP server in Heroku would seem like using a wrong tool for it. While it's good if that could be possible, but any kind of approach with it would be not straightforward. Much cleaner solution for those who rely on Heroku is simply to find another host where to run their XMPP server.

JH

Jonne Haß Sun 12 May 2013 1:41PM

Just to clarify what agreeing to this proposal means: "The WebSockets protocol is not yet supported on the Cedar stack." https://devcenter.heroku.com/articles/using-socket-io-with-node-js-on-heroku

It's also impossible to open arbitrary ports, so running an in process XMPP server is also impossible.

Thirdly I wouldn't consider requiring a third party XMPP server as "working on Heroku".

FS

Florian Staudacher Sun 12 May 2013 6:26PM

long-polling/comet! ;)

JH

Jonne Haß Sun 12 May 2013 6:28PM

I don't think our issue is S2C. Our problem is S2S.

S

Shmerl Sun 12 May 2013 6:50PM

Jonne Haß: Then clarify please what kind of options for XMPP server are possible on Heroku? From what you said, it sounds like they are non existent. So what is the poll about then?

SM

Seth Martin Sun 12 May 2013 7:49PM

Although it's not ideal to trust yet another party with your data, third party BOSH can also be used for the few XMPP users that don't have BOSH. IMO the ideal solution here would be a simple Heroku compatible Jappix mini type plugin like Friendica uses.

JH

Jonne Haß Sun 12 May 2013 8:27PM

That's what I want to clarify. I've not come over a decent solution that allows using XMPP on Heroku without addtional pain/costs for either the users or the maintainer. From my current knowledge it's impossible. But then a large percent of our userbase is on Heroku, one can only guess the numbers, but I'd be surprised if it's below 30%. And we can't say these 30% are on a well maintained and committed pod currently, sadly. While I didn't read every word in this discussion I saw a clear tendency to use XMPP, possibly over Websocket or BOSH. Again this is not possible on Heroku.

So we need to decide if we want to further seek for a solution that works on Heroku or allow one that doesn't work well there. Or we won't move forward in this discussion. That's what this poll is about, support chat on Heroku or not.

(That "possibly using XMPP is in the discussions title is probably a bit confusing and should probably be changed. Nonetheless it's an option and not decided yet.)

S

Shmerl Sun 12 May 2013 9:57PM

If supporting Heroku means not using XMPP for the chat - then I'm voting No. My Yes vote was only for the effort to find how to use XMPP on Heroku.

S

Shmerl Sun 12 May 2013 9:59PM

I'd propose to create a poll for using XMPP as a solid base for chat protocol then. And see what people say.

S

Shmerl Sun 12 May 2013 10:08PM

I agree with what Seth Martin said:

I'm not interested in any other solution than
XMPP because I want to take my chats with me
when leaving the browser. I hardly consider it
screwing Heroku users by requiring they use a third
party XMPP server if they want to chat.

JH

Jonne Haß Sun 12 May 2013 10:31PM

@shmerl Please edit your vote then.

TS

Tom Scott Mon 13 May 2013 12:40AM

@rekado blather looks interesting...honestly i had no idea anything other than xmpp4r existed as a library. wasn't too excited about using something that hasn't been updated in 3 years. ;-)

I imagine this whole XMPP feature would look something like the setup for an email server. In which you tell it what server to connect to, and what username/password to use. You can also perhaps enable or disable the "sync accounts" feature which would basically create an account on the XMPP server according to XEP-0077. This would enable users to use their existing XMPP server on Diaspora IM.

I think what most people here really can agree on is that they all want a web-based chat UI inside Diaspora. For some, this may mean a secure XMPP server for sending messages that they control. For others, it might mean just hooking up their jabber.org account and simply using the UI. We should make each option available.

JR

Jason Robinson Mon 13 May 2013 6:54AM

Loomio isn't taking my vote. Though to me the options are broken. XMPP will be split in no and abstain votes, while non-xmpp gets all the yes votes. I think we should count no/abstain against yes, the options are not well thought out.

Anyway, seriously, are we going to plan the feature set of Diaspora* (free, open source, privacy aware etc) on the whims of one proprietary company? If there is a better option than XMPP then let's hear it - but this HDD (Heroku Driven Design) talk is just mad.

JR

Jason Robinson Mon 13 May 2013 6:55AM

Of course we should allow pods to run on Heroku as easily as possible but it should not control core functionality design.

JH

Jonne Haß Mon 13 May 2013 8:54AM

No problem in counting the majority as absolute one, i.e. not reached if not > 50% of all votes are yes. In fact that was my original thinking.

I don't want to fight for Heroku. I want to know whether to support it or not. I see the discussion stuck otherwise.

D

diasp_eu Mon 13 May 2013 11:26AM

Free at herokus means a starter database with upto 10k rows? Because of the current federation system and duplicate posts you reach 10k very soon, hence you need to pay for running diaspora on heroku. Who is running diaspora on heroku for free?

I know joindiaspora.com runs on heroku but why not using an external xmpp server on another machine?

We should find a solution for chat, not a solution for heroku!

RF

Rasmus Fuhse Mon 13 May 2013 12:30PM

Even if heroku is not allowing XMPP, our decision is not: XMPP or not XMPP. We could also implement a fully working XMPP-chat with a per pod option to disable XMPP-chat for the pod/it's users. Users of the pod wouldn't be able to chat. But that is due to limitations of the crappy Heroku settings. Users of better servers will be able to chat with each other.

We shouldn't let Heroku dictate us how we are using our network. But on the other side we should create our software in a way that it runs even on low-profile machines. We don't need to implement an evil XMPP-hack for Heroku, but we need the option to run a non-XMPP-diaspora-account.

CG

Christian Giménez Mon 13 May 2013 4:18PM

Then, I don't get very well the idea of the Yes/Abstain/No options. What does these means then?

If yes means implement it on Heroku, then I have to disagree or abstain... Heroku is no important mather as long as XMPP works. Unless we agree that this is just a start until we decide another better/free/descentralized alternative.

We have to consider that we shouldn't limite the people runing pods to use Heroku. I mean, if I want to start up a pod, I can do it in my own work/home without any Heroku's connections.

I can see that we all agree to use XMPP. Is a semi-descentralized protocol and that is atractive! That is cool! :-)

ST

Sean Tilley Mon 13 May 2013 6:58PM

@jasonrobinson I don't think it's so much about bending to the "whims" of Heroku, so much as it is the reality that many Diaspora pods are already hosted there. Crippling a featureset for users on a specific host is just as bad as bending to the whims of that host in regard to development.

I think going forward, we need to consider alternatives to Heroku, if we really want to encourage podmins to move over to be able to benefit from chat. I have heard that OpenShift has some promise, and @jonnehass did some work a while ago making diaspora work with OpenShift.

Basically, if we want to develop XMPP-powered chat and not worry about Heroku, we should come up with a good alternative to it that can better suit podmin needs in regards to these kinds of features we want to do. Otherwise, we'll need to find something else that works for everybody, regardless of what host they're on.

JR

Jason Robinson Tue 14 May 2013 6:42AM

@seantilleycommunit implementing a chat solution that is optional but doesn't work on Heroku will not cripple the existing user base. They just will not get the new feature.

I really do believe we should not base any design on a single hosting solution. Be it Heroku, Openshift or any other. Let's find the best solution out there, make sure it works on as many platforms as possible and then decide on that. Be it XMPP or something else.

TS

Tom Scott Thu 16 May 2013 5:10PM

@diaspeu you're misunderstanding how that works. the database doesn't lock, it merely begins deleting old records. perfectly fine for a social network that is more about what's happening right now than what happened 5 years ago. for example, what i wrote on Facebook 5 years ago is not only meaningless but terribly uninteresting to me today.

As I've said before, and I will say again, if we do this I volunteer to either code it or head up the team that's tasked to make this all work.

D

diasp_eu Thu 16 May 2013 7:00PM

@tomscott I got it, but at diasp.eu it takes 6 days to get 10000 posts (approx. 6000 from joindiaspora.com), means my last post, 7 days ago, will be deleted :-) Take a look at https://diasp.eu/stats and see how the federation system fill up my db.

TS

Tom Scott Thu 16 May 2013 7:01PM

@diaspeu haha makes sense, you have a public pod. mine is private and just for me. having a free heroku instance run your pod makes sense for people who don't use Diaspora that often and just want to control their own data. in my case, i've always just liked playing around with the code, so it's nice to be able to use my own changes.

D

diasp_eu Fri 17 May 2013 7:09AM

@tomscott Your pod at https://s.psychedeli.ca/posts/1 running an old version and I'm sure that your workers are dead because you don't receive any posts from other pods.

If you run diaspora as intended with workers to receive posts from other pods you db will fill up quickly and thats the reason why this will not work on heroku for free :-)

F

Flaburgan Fri 17 May 2013 8:03AM

(juste about the "WebRTC is not available yet", Mozilla put it on by default in the last beta, so it'll be in stable in 6 weeks.)

F

Flaburgan Fri 17 May 2013 8:43AM

I really do believe we should not base any design on a single hosting solution.

Everyone will agree that. But I have another point of view: I'd like to see Diaspora light. I'd like to run it on my Raspberry Pi, with only a "apt-get install diaspora" that the package team will provide. And for that, I have to be the lightest solution possible for the chat feature: my pod should do the minimum. For me, integrate an XMPP server in Diaspora* is not a good solution because Diaspora will become too heavy.

Keep things simple: a setting for the podmin to indicate an XMPP server which will be used by Diaspora*. This server can be installed in the same server, in another one, or maintained by someone else like jabber.org, but will not be integrated inside Diaspora, which means you are not obliged to install it in the same host.

Does this solution fit with Heroku? I don't know.

The other solution, more powerful, lighter, and which works on Heroku, is to use WebRTC. But it's less mature, cannot be used outside browser for the moment, and is not linked to our already existing XMPP account (can't talk to gtalk, etc.)

S

Shmerl Fri 17 May 2013 4:03PM

@Flaburgan: I think we should stick to interoperable protocols as a priority. So creating custom made protocol through WebRTC is a bad idea by default. "Lightness" can be achieved by not requiring XMPP server to run on the same host as Diaspora.

I'm sorry about Raspberry Pi, but it's completely not suitable to run an XMPP server, let alone any other serious server with multiple concurrent connections, but I doubt anyone would want that anyway, so in such scenarios D* pods will use remote XMPP servers.

F

Flaburgan Fri 17 May 2013 5:26PM

@shmerl i completely agree, this was my point: I don't want to integrate an XMPP server to Diaspora*, I want to make it possible to run in another host.

JR

Jason Robinson Fri 17 May 2013 6:55PM

I think a Rails Diaspora* will always be not available on a Raspberry Pi, due to memory use ;)

DU

[deactivated account] Sat 18 May 2013 10:29PM

@Flaburgan: it's probably not wise to mix this proposal with the idea of running Diaspora on ARM. There are quite a few other problems you'd have to solve first (e.g. database) before chat would become a problem. If you'd run the database server on a remote machine for performance (leaving the Pi as a control centre only), then the same thing should be done with an XMPP server. Just put it on a separate machine in the same network as the Pi.

JH

Jonne Haß Sat 18 May 2013 10:37PM

@rekado running Diaspora on Pi for a single user pod is entirely possible. It's slow but possible and yes, I kept the DB on the Pi too. But we're really getting offtopic here.

DS

Dennis Schubert Thu 23 May 2013 7:49AM

Just in case it's not clear for everyone: if somebody wants to start implement a instant messaging functionality based on XMPP, I'd be able to mentor the XMPP part of the whole process. Just contact me..

D

diasp_eu Fri 24 May 2013 10:29AM

What are the next steps?

RF

Rasmus Fuhse Fri 24 May 2013 1:04PM

I guess, it's quite clear how the chat should be implemented now. Next step should be to find someone who implements it. It's your proposal actually.

IGM

Ivan Gabriel Morén Sat 25 May 2013 4:51PM

Is it too whacked to think that a hack like the auto-update via greasemonkey could do it alternatively? I'm thinking of a simple checkbox at the bottom of the conversation view, so that you can let it check for new messages every tenth seconds or so if you're having a hot debate with someone?

RF

Rasmus Fuhse Sat 25 May 2013 5:39PM

Well, it's not a real hack. It would absolutely work. but still we had to implement a federation and stuff. And when we want to do so, we should most likely implement XMPP instead, because it's an open standard.

T

theradialactive Sun 26 May 2013 4:03PM

I prefer XMPP, because we could maybe connect it to WebRTC, so we have a really big feature coming.

http://hancke.name/jabber/webrtc-xmpp-summit-13ph.pdf

TS

Tom Scott Mon 27 May 2013 4:41PM

@diaspeu the workers aren't running because the SSL cert is fucked up right now. I never really had too many friends on Diaspora in the first place so my profile never got much activity even when it was on joindiaspora.com…which is why I thought a free Heroku instance would be sufficient for a presence. I really mostly use it to test stuff that I'm working on personally.

I'll try and get the SSL cert working again to see if running it on a free Heroku instance is as bad of an idea as you say it is. =D

TS

Tom Scott Mon 27 May 2013 4:42PM

@dennisschubert I would love to work on this with you. are you around much on IRC?

DS

Dennis Schubert Mon 27 May 2013 5:28PM

@tomscott cool! sure thing, my nick is 'denschub'. please note I'm living in CEST (but I'm usually working all night long, so that should be no big deal). alternatively you can reach me on XMPP, denschub@dsx.cc

DS

Dennis Schubert Mon 27 May 2013 5:29PM

(eh, denschub on freenode, of course.)

D

diasp_eu Mon 27 May 2013 5:31PM

+++ Thanks to all of your comments and support in this story. Now we come to the important point, the implementation. Let us work together and make diaspora a better place! +++

DU

[deactivated account] Mon 27 May 2013 10:36PM

@Dennis, @tomscott: what are the components you'd implement? Are you going for a full XMPP client in JavaScript that registers with the local XMPP server (e.g. via BOSH) or a minimal connection between browser and dedicated XMPP backend (e.g. websockets that push messages through to an XMPP server component which is connected to an XMPP server---local or remote---through XEP-0114)?

DS

Dennis Schubert Tue 28 May 2013 7:46AM

Although I have not talked with @tomscott yet, I think we're going to implement a full XMPP client based on Strophe.js. That will take a bit more work than just passing the stanzas through some other channel (as we have to write a little auth provider to authenticate with your Diaspora user credentials), but that's the cleaner solution. IMO.

JH

Jonne Haß Tue 28 May 2013 10:06AM

We should also consider just storing a JID and a password and running an open BOSH proxy like https://github.com/dhruvbird/node-xmpp-bosh as a fallback setup.

S

Shmerl Tue 28 May 2013 4:28PM

Full XMPP client can use Websockets as well which is preferable over BOSH. But some XMPP servers are lagging behind with supporting Websockets (for example open source ejabberd didn't support them the last time I checked, but commercial one did). I'm not sure if that changed recently.

DU

[deactivated account] Tue 28 May 2013 8:53PM

@shmerl:

But some XMPP servers are lagging behind with supporting Websockets

I don't think it's something they even should support. It's not like that is defined in an XEP somewhere. (There's a third-party draft for XMPP over websocket, but it's not anywhere near a standard, so one can't expect XMPP servers to support websockets natively.)

To shuffle messages from the frontend to some Ruby backend via websockets is trivial. The Ruby backend could easily manage the necessary state (e.g. Diaspora authentication and other plumbing) which would otherwise be rather awkward in JS (the Ruby process could talk to the db directly, unlike some client-side JS which must take a detour through an HTTP API).

(I've been using blather over the past weeks to turn Libertree into an XMPP server component.)

S

Shmerl Tue 28 May 2013 9:03PM

It's good if they support Websockets the same way it's good when they support BOSH. I.e. it's not really necessary for pure XMPP over TCP/IP, but for web browser solution it is necessary, since they have security limitations and can't open free socket communications the way standalone clients can. Therefore some additional support from the XMPP servers is needed (whether it's BOSH or Websocket) to enable communication with browser based clients.

I meant direct communication with the XMPP server over Websockets, rather than with Ruby backend. It's more interesting, and there are efforts in that direction.

S

Shmerl Tue 28 May 2013 9:07PM

For the reference, the draft you mentioned is probably this one: https://datatracker.ietf.org/doc/draft-moffitt-xmpp-over-websocket/

TS

Trolli Schmittlauch Wed 29 May 2013 7:16PM

Another thing: I think everyone here is of the opinion that real decentrability is only possible if one can simply switch to another server and take the data and contacts to the new server. This feature has jet to be implemented into Diaspora, but it's possible. But what about XMPP? I experienced that switching servers is not so easy when using XMPP.

DU

[deactivated account] Wed 29 May 2013 10:37PM

@Trolli: XMPP is just a transport protocol. If you were to implement the whole thing as a server component, things like data migration (as well as everything else) would have to be handled by the server component. If you tie the implementation to a single XMPP server instance, however, you are subject to its implementation defects.

DU

[deactivated account] Wed 29 May 2013 10:40PM

@Shmerl: yes, that's the draft. Note that this is not a draft published on the XMPP site and also note its status.

If you were to develop a solution that expected an XMPP server to support websockets, you'd make the feature incompatible with most XMPP server implementations out there.

TS

Trolli Schmittlauch Tue 11 Jun 2013 6:27PM

@dennisschubert : There is also ongoing work for Jingle support in Strophe.js https://github.com/ESTOS/strophe.jingle
evertheless, it still seems to need Bosh

DS

Dennis Schubert Tue 11 Jun 2013 6:46PM

@trollischmittlauch strophe.js will always need BOSH. You cannot create a TCP stream in your browser (yet.) But we'll set up a little BOSH proxy shipped with Diaspora for all those non-BOSH servers...

@tomscott what's the status? ;)

S

Shmerl Wed 12 Jun 2013 3:51PM

@trollischmittlauch: Great, so Jingle over WebRTC is coming at last.

@dennisschubert:

strophe.js will always need BOSH.
Not really - as discussed below, it could use WebSockets which is even better, but it's not yet standardized for XMPP so BOSH is still preferred.

TS

Tom Scott Fri 14 Jun 2013 9:08PM

So @dennisschubert and I have decided to build auth modules for prosody and ejabberd in order to have the XMPP server auth with Diaspora and not have to maintain 2 synced databases of users. Instead, the XMPP server will query the Diaspora pod for whether a given username/password exists, and we keep all user data in one DB.

In the coming weeks, Dennis and I will be discussing how to best implement the Diaspora-side API that deals with XMPP. Since we're writing our own auth module, we are going to start with just POSTing to the Devise user_sessions endpoint, and evaluating the response.

On the client side, I'll be implementing an XMPP client from within Diaspora using strophe.js and BOSH. I know BOSH, that's why I'm using it. We can also give WebSockets a shot but I'm leaning towards the "tried-and-true" for this particular feature. The first iteration will probably not have configurable usernames, but in the future I want to add that so you can connect the web client with any XMPP server, if you just want to use diaspora's chat client. I'm not sure how that will work on pods that don't have an XMPP server, since we probably want to remove any UX related to IM on pods that do not support such a service.

Thoughts and comments would be much appreciated! <3

ST

Sean Tilley Sat 15 Jun 2013 2:28AM

Perhaps a temporary workaround would be to include a configuration flag such as IS_HEROKU, that way if a pod couldn't support XMPP, the interface and related links and buttons for chat would simply not appear at all.

ST

Sean Tilley Sat 15 Jun 2013 2:31AM

Also, out of curiosity, what is the current state of WebSockets? I seem to remember Diaspora supported it ages ago, but it had been shut off for one reason or another.

TS

Tom Scott Sat 15 Jun 2013 5:50PM

@seantilleycommunit definitely. since XMPP chat is an optional feature, the interface won't be loaded unless you configure an XMPP server. when the feature is in master and pods update, nothing will happen. each pod who wants to support this will have to configure their own XMPP server.

TS

Tom Scott Sat 15 Jun 2013 5:52PM

additionally, at this time we are only supporting ejabberd and prosody, since @dennisschubert knows both Erlang and Lua. if anyone wants to contribute an auth module for OpenFire that would be super-awesome. I guess we'll host these projects on the main diaspora organization in GitHub, so people can easily find the plugin they need.

DU

[deactivated account] Sun 16 Jun 2013 2:28AM

I guess I'm repeating myself, but I'd like to warn (again) against developing against particular XMPP server implementations. It's work that you don't really need to do to achieve the goal, you have to track more upstream projects, and it will only work for selected versions. It also means that one cannot connect one's Diaspora installation to an existing XMPP server installation without installing your third-party auth module.

Of all possible solutions this seems to be the ugliest.

DU

[deactivated account] Sun 16 Jun 2013 2:40AM

(Wow, I just used /again*/ for three times in the same sentence.)

SM

Seth Martin Sun 16 Jun 2013 6:38AM

@tomscott "The first iteration will probably not have configurable usernames, but in the future I want to add that so you can connect the web client with any XMPP server, if you just want to use diaspora’s chat client."
I beg you, please don't forget that it should be a high priority to accomplish this ability. I don't think that there are many people that want to have a second XMPP account just for Diaspora contacts. I know... one step at a time. I commend you and @dennisschubert for taking on this project and imo it appears you are making the right decisions.

With that said, I'm glad that I just revisited this discussion because I'm prepared to install an XMPP server on the same machine that hosts D*. You just helped me decide on Prosody. Something that's not yet clear to me is whether I will be able to use node-xmpp-bosh instead of Prosody's local BOSH. I wish to have the ability to provide an open BOSH for those without. Will this be possible?

SM

Seth Martin Sun 16 Jun 2013 6:55AM

@tomscott also, if I were to set up my XMPP server now add an account for myself with the same username/password as my Diaspora account, would I be able to use this account with the first iteration?

TS

Tom Scott Sun 16 Jun 2013 3:40PM

@rekado this actually came up in our discussion. i would be in favor of doing something like this: http://xmpp.org/extensions/xep-0077.html but, it was brought up by @jonnehass and @dennisschubert that keeping two DBs in sync (the XMPP server's user database and Diaspora's database) might be more difficult. What do you think?

@sethmartin from what I said in that post, no you wouldn't be able to at this time. but, we're still in the planning process. if it turns out that keeping 2 separate DBs of users is easier than doing the auth module thing, this may change.

VC

Vittorio Cuculo Mon 17 Jun 2013 3:00PM

Woah! I've just read through the entire discussion in one go. And it's nice to see that after 2 years this is still a most wanted feature for D*.
I read that you've decided to adopt XMPP, and since I've realized a working implementation of the chat/videochat feature for D* (2 years ago) I may give you some advice and helps developing it.

Server side

As @tomscott said, keeping two DBs in sync might be difficult as well as expensive ($$$).
But the XMPP server needs to access to D* database in at least 2 occurences: authentication and friends list.
So here are the solutions I've used, and I hope they may help you.

Authentication

Ejabberd offers as auth_method an external option which uses an extauth_program to check if credentials are correct.
The extauth_program should follow these specifications.
You can find my implementation here and other info here
This solution allows to use the xmpp account also with external clients.

Aspects/rostergroups

XMPP provides "rostergroups" that correspond to D* "aspects", and "rosterusers" that correspond to your friendlist.
To keep these connections my solution was to create two DB views to get rostergroups and rosterusers, that, by definition, they are updated when an edit is done on the tables.
But I'm not sure if this is the best solution (at least it was for me 2 years ago).
You can find it here, in particular rows 161-162.

Other

Other minor configurations are required to ejabberd.cfg file. Here's an example which is the same of the original configuration file but edited in these lines

Client side

Reading this discussion, I think you have already carefully considered all the possibilities. In my implementation I've used BOSH too, because the websocket module for ejabberd was still unstable (I don't know the current status).
The client side was, of course, made by two JS files, one which implements the core functionalities (that you can find here ) and it uses Strophe.js library to connect to the XMPP server. And the other file implements the UI here , and it's based on mini.jappix.

Problems

  1. Consider that people may want to use their existing xmpp account
  2. Consider different types of XMPP server
  3. Consider that you may need to open 3 different ports (c2s for non-web clients, s2s, bosh).

That's all folks!
I really hope this might be useful for you, and can avoid you to reinvent the wheel.
For any need you can ping me here or on D*.
Oh, and thanks to @diaspeu for bring me here :)

VC

Vittorio Cuculo Mon 17 Jun 2013 3:02PM

G

goob Mon 17 Jun 2013 3:08PM

Whaouh, that's great. Thank you Vittorio.

TS

Tom Scott Mon 17 Jun 2013 6:18PM

Awesome @vittoriocuculo, so glad to have you participating in this discussion as I believe you were the one who made the initial PR with a basic implementation of XMPP on Diaspora. You fuckin rock!!

I think we could all look at Vittorio's implementation of this as an example for how I'd like features to be introduced into Diaspora. Someone forks the projects and takes the initiative to figure out the codebase and make a major change without deliberating for a long time on how it should be implemented. Even though his patch wasn't initially accepted, we are clearly learning from his experience and I'm intending on reusing as much of his code in the new PR as possible.

So, double thanks to you Vittorio! I'll do some more prototyping this week and check out your fork to get inspired.

DU

[deactivated account] Wed 19 Jun 2013 11:34AM

@Tom Scott: you don't have to maintain two databases when you cheat by implementing an XMPP server component. A server component can connect to any XMPP server with its own domain. What's great about server components is that they can author messages for any user on their domain. This means that the server component could directly use the Diaspora database and map this to XMPP concepts. The XMPP server is only used for transmission and reception of stanzas; any stanza addressed to the server component's domain is not processed at all and directly passed to the server component.

In my opinion this is a much cleaner implementation (which is why Libertree and Buddycloud do it this way). You'd have to implement a mapping between aspects and rosters and wire up the stanzas you want to support. This is considerably less effort than developing parts against existing XMPP server implementations and it's much more robust as it's completely independent from any XMPP server implementation.

A server component is easily written with blather; it runs in its own EM event loop and could be spawned during application initialisation. It's self-contained and can be implemented in 100% Ruby.

DS

Dennis Schubert Wed 19 Jun 2013 12:43PM

I' enjoying your discussion...

Just letting you know the estos GmbH just released a strophe.js plugin to support jingle (and some fun with WebRTC if available): https://github.com/ESTOS/strophe.jingle

S

Shmerl Wed 19 Jun 2013 3:47PM

rekado: How easy would it be to add Jingle support to such component? It might be a big headache once you decide to do it, in comparison with working with major servers like ejabberd and others.

TS

Tom Scott Wed 19 Jun 2013 9:07PM

@rekado that sounds like the best of both worlds. is it too good to be true? :)

alright so can you write that server component? or will we need to find another volunteer? i would do it if someone takes the client-side work off my hands, because if we both work on each part at the same time and communicate properly, we'll have this done pretty quickly.

DU

[deactivated account] Wed 19 Jun 2013 11:53PM

@tomscott I'm afraid I won't have enough time to work on this :-/

It is fairly simple but my next months are pretty much (over)booked and I wouldn't like to delay this feature.

XMPP server components are simple, which is their biggest advantage and their biggest disadvantage. They allow you to use XMPP servers to handle communication, but they don't actually do anything you haven't implemented. Handling chat messages and supporting presence stanzas is almost trivial. If you want to support extensions to the protocol, however, this can be more difficult. (But it's pretty difficult/messy to link an independent extension implementation in some XMPP server to Diaspora anyway. I still hold that the bare XMPP server component is a cleaner platform to work with.)

@Shmerl: you are right, Jingle is not supported out of the box. Looking around there doesn't seem to be a dedicated Ruby library for jingle. The XMPP part of Jingle is pretty easy to implement; after all, these are just normal stanzas with payloads to negotiate a connection (data transmission itself done outside of XMPP). Actually establishing certain connections and supporting various protocols is more difficult, but I'm sure there are Ruby libraries for that. The only missing piece would be the negotiation according to XEP 0166.

TS

Tom Scott Thu 20 Jun 2013 1:01AM

@rekado looking at the docs for blather, it seems like a pretty simple DSL. I could probably hack it, but it may take a little longer to get the client side portion out unless I can get some help with that stuff. I'm also not interested in extending the XMPP protocol…supporting basic messaging and authentication between the app and an XMPP server is just fine. so I think the server component written in Blather would be perfect.

i'm thinking about making this a separate Rails::Engine. this way, people can include the gem into their Diaspora installation, like a plug-in, and it would provide the XMPP server component as well as the front-end chat features. this allows people who can't or don't want to run their own XMPP server to simply disregard all of the code for this feature. a mountable engine works seamlessly with a Rails app, as if it were part of the app itself, so I think it'd be a perfect way of encapsulating this feature.

S

Shmerl Thu 20 Jun 2013 4:06PM

rekado: I think it's better to avoid implementing any server components, since it's reinventing the wheel and will eventually simple fall behind and become a big mess. If possible, Diaspora should handle only client side, where server can be anything external.

The issue is, what and how do you want to define as a list of users and groups in the context of Diaspora in such case. That's as already mentioned below is something to figure out. It's better to solve this issue once, rather than trying to catch up with XMPP server development, any time new stuff is added to that. Just estimate the effort.

DU

[deactivated account] Fri 21 Jun 2013 11:12PM

@Shmerl: "reinventing the wheel" <--- no, everything is wrapped up nicely in a library.

If Diaspora only handled the client-side there would be no integration of the chat feature. There would be no advantage of using the built-in chat client over an external chat client.

A server component is not the same as developing an XMPP server. If you write against specific server implementations to gain some degree of integration, you'd still have to figure out how to map the server's features to Diaspora features. The difference is that in the case of writing against plugins you don't have library support and you can't write in a generic fashion. You'd have to duplicate your efforts for every server implementation that you want to actively support.

XMPP is no magic. You can't just write once and reap all benefits of all XMPP extensions that happen to be available on a given server. Not if you want to go beyond embedding a generic client in the browser window, which is the most unsophisticated and least helpful approach to adding XMPP chat support.

S

Shmerl Wed 3 Jul 2013 5:55PM

FYI: May be this will be useful for something: http://candy-chat.github.io/candy

F

Flaburgan Fri 5 Jul 2013 8:58AM

With prism, many people want to leave gmail. But they use the chat and miss it. Having a chat in Diaspora is one argument more for adoption.

What's the progress on this guys?

DS

Dennis Schubert Sun 14 Jul 2013 3:36PM

@tomscott anything new, dude?

DU

[deactivated account] Mon 29 Jul 2013 12:14PM

I think this is perfect idea. I use XMPP in other app for notifications and is it very usefull for me.
The chat in diaspora is awsome idea.
But it must be standalone chat, no other server. Diaspora must respect privacy.

A

aj Mon 29 Jul 2013 11:23PM

my suggest is keep it simple

a css/javascript client dropped into the template for logged in pages

and a place in the user profile/db where user can share his jabber id

i'm testing JWchat free code and have ejabberd installed to my pod, which works well if you want the jabber id the same as the d handle, but it should not be required to install an XMPP server, and the client should be able to use any xmpp server

F

Flaburgan Tue 30 Jul 2013 8:02AM

+42 for @aj

DU

[deactivated account] Tue 30 Jul 2013 9:59AM

I think two steps (diaspora accont and jabber id) will destroy the chat idea in diaspora.
Why? Users don't want communicate difficult. They want login & chat. No find or create jabber id etc. Only login and chat. Then i think define jabber in profile is bad idea, the better one is do it simple for user.

F

Flaburgan Tue 30 Jul 2013 12:38PM

@maxsamael you need to read the discussion to understand why we want to do that.

DU

[deactivated account] Tue 30 Jul 2013 2:50PM

@flaburgan thak you, i will check it

S

Shmerl Tue 30 Jul 2013 4:04PM

@max samael: Different users want different things. You can't say for all "users want only this" or "users want only that".

DU

[deactivated account] Tue 30 Jul 2013 9:29PM

Sure, but what is point of decide? Who take the point?
I not, you not, nobody? Thats not true, someone take it. Than say what you want and I will say it too.
I don't say it must be there, but i say it can be there and i think im not alone. Then I say many users. Sometimes is update good and sometimes bad, but who decide about it? Say what you want and many people can listen and decide. I will decide too. And what do you think now? Are you free in speak. I sad yes, we are free and we wont be free. I think diaspora isn't there for north korean government, but for people. I sad for realtime chat YES.

ST

Sean Tilley Tue 30 Jul 2013 10:48PM

@maxsamael Our decision-making process is based on the idea of rule-by-consensus. Basically, we get a feel from our developer and user community as to how we feel about any given idea: the technical implications, how a feature might work, what kind of work would need to be done, and so on and so forth. The point of this process is to see what ideas are:

  1. Practical to put into motion
  2. Have enough volunteers to work on them directly
  3. Doesn't cause huge potential problems in our code
  4. Is something that everyone voting wants.

Everyone has a chance to voice their opinions here. :)

DU

[deactivated account] Wed 31 Jul 2013 7:34AM

@seantilleycommunit Agree, thank you

G

goob Wed 31 Jul 2013 11:01AM

@maxsamael I recommend you read the discussion further down this page to see why XMPP was agreed on as the standard to use in a Diaspora chat feature. All the discussion is there to see. It was a long discussion and many different options were evaluated before XMPP was agreed as the best one for Diaspora!

N

NicoAlto Wed 31 Jul 2013 11:15AM

Hi, this little post just to say that I agree with the idea of chat feature, and the choice of XMPP.

DU

[deactivated account] Wed 31 Jul 2013 11:41AM

@goob Thank you, I will check it detaily

F

Flaburgan Fri 2 Aug 2013 2:52PM

@tomscott @dennisschubert any news? Can we see your work somewhere?

DS

Dennis Schubert Fri 2 Aug 2013 3:03PM

@flaburgan there is no work since I had no contact with @tomscott about this topic so far...

RB

Roger Braun Thu 15 Aug 2013 8:55AM

As far as I understand it, a xmpp server component is more or less just a client that can act as a whole domain, so if we would use xmpp server component, we could not use a standard xmpp javascript client in the frontend, right? This would suck a bit if we would want to support non-diaspora jabber accounts later on.

RB

Roger Braun Wed 21 Aug 2013 10:38AM

Any news on this?

F

Flaburgan Wed 4 Sep 2013 1:35PM

I think this is the next really wanted feature, with pod migration. @rogerbraun no news, if you want to give it a shot, you're more than welcome :)

RB

Roger Braun Wed 4 Sep 2013 5:00PM

I really want to get started with this, but I dont really see xmpp as a good first step. I think a more conservative approach of extending the existing messaging system would be a better and much easier start. This could be connected to xmpp with a server component later on.

JH

Jonne Haß Thu 5 Sep 2013 9:37AM

@rogerbraun that won't ever yield anything near realtime communication.

RB

Roger Braun Thu 5 Sep 2013 9:39AM

I don't see why not. Messages can be displayed in a FB-like chat window and the client can be notified via websockets.

JH

Jonne Haß Thu 5 Sep 2013 11:53AM

The latency issue isn't client <-> server but server <-> server

RB

Roger Braun Thu 5 Sep 2013 2:53PM

Ah, OK. I did not really think about the federation aspect...

RB

Roger Braun Thu 5 Sep 2013 3:48PM

Okay, how about this then:

A xmpp server is setup somewhere to handle the domain the pod runs on. On user account creation (or whenever, maybe after asking the users), the pod automatically creates an xmpp account and a password the user by registering with the pod via xep-0077(http://xmpp.org/extensions/xep-0077.html). Once a user is registered, all the pod does is give the necessary credentials to a client-side xmpp client (jappix or whatever).

What do you say? This would only involve writing the code to include a client and the code to do in-band registration.

DS

Dennis Schubert Thu 5 Sep 2013 4:37PM

We discussed that in the past and decided that this is not the way to go.

RB

Roger Braun Thu 5 Sep 2013 5:10PM

@dennisschubert I knew I did not come up with this on my own ;-) Sorry, the last time I read the discussion completely was some weeks ago, so I forgot parts of it.

Still, after re-reading the whole thread, I can not find any argument against this approach except that it might be hard to synchronize user lists. Is that all that would be bad about this solution?

G

goob Wed 18 Sep 2013 10:55AM

As it appears from the silence that @tomscott is unable to work on this at the moment, would anyone else like to volunteer to work with @dennisschubert on it?

G

goob Wed 18 Sep 2013 4:01PM

Of course if I understood the first thing about XMPP, I'd volunteer myself!

F

Flaburgan Wed 18 Sep 2013 7:47PM

@goob it looks like @rogerbraun is interested to work on it ;)

RB

Roger Braun Thu 19 Sep 2013 7:58AM

Yeah, I'm still looking for an explanation why the xmpp-server + diaspora-server + automatic registration is a bad idea. It seems to be the easiest to implement.

DS

Dennis Schubert Thu 19 Sep 2013 4:12PM

@goob @rogerbraun I'd help you with the XMPP-parts, all you'd have to do is writing the modules :) If you want to, we can talk about it on #diaspora-dev later (in 3-4 hours) or you could add me on XMPP, denschub@dsx.cc and I'll answer your questions as soon as I have some time left. :)

F

Flaburgan Fri 20 Sep 2013 4:51AM

Just wanted to know something: is XMPP a secure protocol? I mean, is it encrypted communication?

If not, what about simply use IRC? I do that with jappix mini on my own website and it works pretty well.

DS

Dennis Schubert Fri 20 Sep 2013 9:18AM

You can encrypt data over XMPP, yeah?!

IRC is a groupchat protocol, you cannot send metadata, there are no contact lists or something similar. IRC is not designed for an application like we need, it's designed for very simple multi user chats, that's it.

JH

Jonne Haß Fri 20 Sep 2013 10:46AM

Uhm, Jappix is a XMPP client not an IRC one.

G

goob Fri 20 Sep 2013 11:01AM

I'd love to help, Dennis, but as Jonne can testify, any sort of coding is not something I'm good at!

F

Flaburgan Fri 20 Sep 2013 10:23PM

@jonne I used jappix mini on my website with Freenode and it worked. But yeah, xmpp is definitely better.

JH

Jonne Haß Fri 20 Sep 2013 10:31PM

MINI_GROUPCHATS = ["geexxx%irc.freenode.net@irc.jappix.com"];

That's an XMPP -> IRC gateway. It's still using XMPP.

DU

[deactivated account] Sat 21 Sep 2013 12:22AM

@dennisschubert XMPP servers establish a TLS encrypted stream between one another for message transmission. Encryption can be negotiated and unencrypted connections can thus be refused.

DU

[deactivated account] Sat 21 Sep 2013 12:23AM

@flaburgan erm, the previous comment was directed at you.

F

Flaburgan Sat 21 Sep 2013 10:13PM

I think if we want to see progress on this, we should open a PR asap. It worked really well with the SPV, stuck for months, then @rogerbraun decide to start working on it and everyone followed. I think we could have the same approach here.

RB

Roger Braun Mon 23 Sep 2013 6:53PM

One big thing that keeps me from working on this is the current state of the frontend, with half of it using blueprint, half of it bootstrap, some parts are backbone, some not, etc... I am not sure that just dropping even more JS and CSS randomly into the page do much good. How self-contained is Jappix?

F

Flaburgan Mon 23 Sep 2013 10:13PM

@rogerbraun we will abandon blueprint, so let's just work with bootstrap (and backbone if needed).

S

StarBlessed Mon 30 Sep 2013 2:53AM

Has anyone taken into account the ability of XMPP servers like OpenFire to allow remote Server to Server sessions? This could act a little like Diaspora federation... Just a thought ;)

C

Crazypedia Fri 25 Oct 2013 1:08AM

Is there, at the moment, any semi-simple way to share XMPP authentication with the local pod's user database? That is, the local pod user uses the same credentials as their diaspora* account to log into the XMPP client?

would a python script make for some suitable bridge to do this?

DU

[deactivated account] Fri 25 Oct 2013 3:46PM

@jacobschleappi That's what XMPP gateways are used for. A gateway is a service that bridges between XMPP and some other protocol.

S

Sakwe Fri 1 Nov 2013 7:01PM

Hello everybody,

I'm new here so it will be my first contribution. I've read this conversation and I'll try to give a solution I imagine.

I agree with the fact that xmpp is best way to manage chat and offer a decentralized option.

So, we have the "xmpp server"(s), the "xmpp client" and maybe some data to synchronise or bridge between D* and xmpp databases.

I don't master Ruby at all but I work for it. I've read a little bit the code of the "Vines" project (https://github.com/negativecode/vines) that offers a xmpp server in a Ruby gem.

I think that with this project, run the xmpp service from the D* database would be pretty easy and "integrated" . It "only" need to hack the storage managment of the "Vines" project to add support to the D* database (users/people/contacts/aspects). It currently uses Nokogiri (and optionaly LDAP for auth) and have a gem that provide the support for storing in sql database. (https://github.com/negativecode/vines-sql)

With this solution, no need of 2 differents accounts on D* and xmpp and the ID in both nodes are automaticaly the same. No need of two databases. The Pod offers the xmpp service directly from the D* database if these gems are installed on the server. Users that host their account on this Pod have access to their new xmpp account in the « Jappix Mini for D* » (for example) or they can connect from another xmpp client to their D*/xmpp account. Their contacts and groups are the same. The D* node become a xmpp node too.

F

Flaburgan Sat 2 Nov 2013 2:04AM

That's a good idea, but it's also heavier than what we tried to be. Linking an external XMPP server is way lighter than integrate a XMPP server inside diaspora*. It would give the choice to podmins to install the server on the same machine, another one, or simply redirect users to an existing XMPP server (jabber, duckduckgo...)

S

StarBlessed Sat 2 Nov 2013 11:58AM

Using an external service requires further action on the part of the user. The whole idea was to make the transition as simple as possible. Jabber (Openfire as an example) only uses minimal memory. I imagine Vines (the Ruby gem) would use only a small amount as well. As for processing power, its not very intensive at all.
I don't think pushing a user to an external service is ideal. We want to keep the pods independent, but allow for federation. Per pod XMPP would make it easy. Both the pod and XMPP server can use the same SSL cert with minimal configuration.
TBH - I think its time to make the choice.
Heroku was a good place to start while there was little funding and only startup pods.
However, there are much more comprehensive solutions now that are funded by people dedicated to the cause. @davidmorley and myself are examples.
I fund my pod out of my own pocket. I don't ask for donations, and I only pay for a server I can already afford.
If the likes of Joindiaspora are on a single Heroku Gear, then I'm pretty certain that its time to upgrade. With the amount of traffic that they have and all.
The Diaspora Project cannot afford to be held back by those who are unwilling or unable to move forward with the times.
Yes, Joindiaspora is the largest of the pods, but it is not the main face of Diaspora. The two biggest places for information regarding D* are the diasporafederation page and podupti.me
Remembering full well that @davidmorley hosts podupti.me himself.
Yes, we may lose a few pods due to funding issues, but what we will gain is momentum. Something this project is sorely lacking in.
If you want to keep Heroku in the loop, then the only option is to code around the limitations of the single Gear. Make the XMPP server an optional install, like a module.
Having an extra few steps on the wiki about how to include something like Vines in your installation wouldn't hurt anyone.
Joindiaspora will still be able to federate, but without XMPP chat. If they want it, then they can upgrade their server to a paid or sponsored system. Otherwise, it would be no different to what it is at the moment.
No harm, no foul. Everybody wins.

F

Flaburgan Sun 3 Nov 2013 10:59PM

@starblessed we already voted about that and we decided to not be stuck by Heroku. What I had in mind saying we have to keep diaspora as light as possible was more about someone wanting to host himself on a raspberry PI for example.

F

Flaburgan Tue 5 Nov 2013 10:18AM

I just wanted to say that, in my mind, having a chat in diaspora became number one in the TODO list. With the NSA scandal, I see more and more people coming on my pod. We are freeing them from facebook and that's awesome. But there is something worse than facebook: google, and especially gmail. To have another address than gmail is easy, but many people doesn't change because of gtalk. Being able to propose an alternative which, besides, allow them to talk with gtalk contacts is awesome. We have to include XMPP chat in diaspora as soon as possible, it will help the XMPP network to grow and become more known outside the geek sphere.

@dennisschubert @tomscott @rogerbraun @sakwe and @vittoriocuculo too if you want to help, I'm sure that if we work all together, the dream can become true! Let's do it!

GL

Guillaume Lakano Tue 5 Nov 2013 5:31PM

About the chat, I can propose my time and knownledges to add the support of WebRTC video/audio.
This is currently only supported inside recent Google Chrome/Chromium and Mozilla Firefox versions.

And if necessary, I can also add a fallback support for others browsers to uses Red5 and Adobe Flash plugin ( but this is a bad idea IMHO ).

S

Shmerl Tue 5 Nov 2013 6:26PM

@guillaumelakano: What is the current state of Jingle over WebRTC support? It was still in development a while ago. Is it already usable to be added to Diaspora chat?

GL

Guillaume Lakano Tue 5 Nov 2013 6:35PM

@shmerl : We haven't to understood what library is used by the browser ( libjingle indeed ), it's all transparent for the developers.

It's already useable yes, I have already done a project for a client with WebRTC. This works between Firefox and Chrome browsers like a charm ! This also works on my smartphone Android with Google Chrome updated.

May be I haven't see it, but currently I don't see any chat available on Diaspora (I'm under diasp.org )

S

Shmerl Tue 5 Nov 2013 6:46PM

@guillaumelakano: It’s not about libjingle, but about support for the XMPP/Jingle protocol over WebRTC.

I.e. imagine such use case:

  1. User A logs in Diaspora chat, and sees contact B there. Contact B is logged in through standalone XMPP client (Pidgin, KDE Telepathy, Jitsi or whatever).

  2. User A initiates a video/audio call, which goes through WebRTC and uses Jingle protocol for signalling. User B picks up the call and they communicate.

In order for that to work, Jingle implementation through WebRTC should be already in a workable state, that’s what I was asking about. I have no idea where things stand now.

EK

Emmanouel Kapernaros Tue 5 Nov 2013 6:48PM

I would like to say that I think the lack of chat in diaspora prevents many people from using it. Also I have noticed an increasing number of people registering on my pod lately. As I see it, it is time to support a secure chat.

WebRTC looks great but if XMPP is more secure I would prefer XMPP.

S

Shmerl Tue 5 Nov 2013 6:53PM

@emmanouelkapernaro: As discussed above, WebRTC is not a substitute for XMPP, they are different level frameworks.

GL

Guillaume Lakano Tue 5 Nov 2013 7:28PM

@shmerl : I don't understood why you ask questions about Jingle. There is nothing to known or to understood about Jingle, it's automatic, as the video codec.

I explain how this could works in Diaspora :

  1. User A - Want to start the camera of User B, he click on the chat link "Ask camera of User B".

2- The chat ask authorisation to User B. If B accept, the chat send a special random unique identifier to User A, and start the WebRTC connection

3- User A receive the random unique identifier and start the WebRTC and attach the connection to this identifier.

That's all !

Nothing to worry about codec or jingle !

And with this technique, we can also permit for a external XMPP program to see the camera with a http link.

GL

Guillaume Lakano Tue 5 Nov 2013 7:29PM

@emmanouelkapernaro : XMPP is for the chat message exchanges. WebRTC is a new HTML5 technologie to permit to exchange binary stream in P2P as the camera/audio.

S

Shmerl Tue 5 Nov 2013 7:34PM

@guillaumelakano: It looks like you are missing some basic facts there. WebRTC does not mandate signalling protocol which applications use to perform media calls. In order to have proper chat with media capabilities, you need the whole stack. Transport layer, presence protocol and signalling protocol. WebRTC is essentially the transport layer. It does not mandate what presence and signalling protocols are used. In XMPP world, presence protocol is the XMPP itself, while signalling protocol is called Jingle (you can read about it). There are other signalling protocols like SIP for example. In order for any client to have any clue how to communicate you need to use some of these protocols, otherwise you'll have zero interoperability.

From the WebRTC FAQ: http://www.webrtc.org/faq#TOC-Why-should-I-use-WebRTC-

WebRTC abstracts signaling by offering a signaling state machine that maps directly to PeerConnection. Web developers can therefore choose the protocol of choice for their usage scenario (for example, but not limited to: SIP, XMPP/Jingle, etc...)

I hope this clarifies things. So, let me ask this again. How does implementation of Jingle over WebRTC fares now, is it even ready?

S

Shmerl Tue 5 Nov 2013 7:41PM

So let's go back to my use case but from different side. User A related to Diaspora opens Pidgin (or Jitisi or etc.) and logs in to the XMPP server. Some contact B of that user is using Diaspora web client at the same time, and they see each other as on line. User A (in Pidgin) initiates video call to user B. Pidgin makes XMPP/Jingle call to user B as proper. The client of user B (inside Diaspora site) uses WebRTC to handle all that. So how will that work exactly?

S

Shmerl Tue 5 Nov 2013 7:47PM

From your example, it looks like you are trying to make your own makeshift signalling protocol. It's not an acceptable approach since you won't be able to communicate with any other client (besides those which use your custom non standard signalling). That's why using Jingle is important.

GL

Guillaume Lakano Tue 5 Nov 2013 8:01PM

@shmerl : Ok, now I understood better your questions.

In my development cases, we only used HTML clients, so we haven't the same problematics.

The audio/video is available between 2 users inside the diaspora web client.

I have no idea about when the WebRTC will support Jingle or SIP capabilities to permit to choose directly the internal protocol to be compatible with external clients as Pidgin.

I said in a previous message "with this technique, we can also permit for a external XMPP program to see the camera with a http link.", it's a technique by opening a popup browser, and not by using the internal support of Jingle of Pidgin

S

Shmerl Tue 5 Nov 2013 8:06PM

Yeah, I see a serious downside in implementing a custom non standard signalling to use with D* web-client <-> D* web-client only and which won't be compatible with any other (non D*) web client or standalone XMPP client for media calls. The proper way to do it - is to implement Jingle over WebRTC when that will be possible. If you can look into that in the context of D* chat - that would be awesome. Thanks.

GL

Guillaume Lakano Wed 6 Nov 2013 8:51AM

@shmerl : In a first step, I think we can make a video/audio only between D* web-clients.

And when this could be possible, we'll upgrade to Jingle to have interoperability with other XMPP applications.

GL

Guillaume Lakano Thu 7 Nov 2013 2:24PM

After some search, I have found a not yet working solution but this should automatically works in future.

There is 2 library ( stanza.io and strophe.jingle ) that permit to uses Jingle protocol inside WebRTC A/V calls.

But Philipp Hancke, the developer of strophe.jingle, explained me it's partially working in Chrome 28, but not anymore in 30.

He advice me to uses the stanza.io library, even if this not working fine today, this should be fixed in future versions of Jitsi / Chrome.

The discussion is here :
https://github.com/ESTOS/strophe.jingle/issues/32

S

Shmerl Thu 7 Nov 2013 4:42PM

Thanks for the update!

PH

Philipp Hancke Thu 7 Nov 2013 9:25PM

so... speaking as the developer of the strophe.jingle stuff and also contribute to the jingle stuff in stanza.io.

WebRTC is bring-your-own-signalling. It's just a media engine which captures data from your webcam or microphone and sends it to a peer. How that is negotiated is left to the javascript. Using Jingle, which is XMPPs way of doing this is one alternative. Check out http://vimeo.com/77289728 for a great introduction to webrtc and why you should avoid building silos (little chance here :-) and use XMPP.

The biggest advantage in using a standard like XMPPs Jingle for WebRTC signalling is of course interoperability. Otherwise you could just buy the "just use our API and exchange an URL and be happy" story that some people are selling in that area.

So strophe.jingle currently kind-of works with jitsi. stanza.io doesn't yet. That is going to be fixed at some point, but fixing and stabilizing interop between strophe.jingle in chrome30 and jitsi is currently a higher priority. In the meantime, XMPP offers a way to detect that a client is not capable of doing WebRTC+Jingle and fall back to sending a URL. Now Jitsi is one of the audio/video clients that drive development of modern protocols. Others, like pidgin, are still lagging behind. WebRTC doesn't allow unencrypted media traffic at all.

There is actually a third jingle implementation in the browser at https://github.com/legastero/jingle-interop-demos and I am expecting another one to join us very soon.
What library you choose is mostly a matter of preference of the guy who develops this. stanza.io is a quite good approach to hide the uglyness of XML (aka "legacy JSON") from web developers by letting them work with JSON objects.
strophejs is an dependable workhorse and there is even a book explaining it. But it's age is starting to show.

F

Flaburgan Fri 8 Nov 2013 8:44AM

Thank you Philip for these precisions. I think that we should start creating a prototype now, what about an IRC meeting to decide who wants to do what? When are you guys available?

GL

Guillaume Lakano Thu 14 Nov 2013 3:48PM

@philipphancke Thanks for sharing your experiences :)
@flaburgan IRC will be difficult to match a convenient hour for everyone (we are not in the same GMT)

I propose to see here, who want to works on the chat.
As explained before, I could give time to add support of video.

About the XMPP chat, I don't known if someone have started something, and how we could attach it to Diaspora website.

About the XMPP chat, we don't need to re-invent the wheel, there is a library that permit to do that easily :
http://conversejs.org/

When the XMPP chat was integrated, I (or any other volonteer) could add the video support with one of the 3 libraries mentioned above.

S

Shmerl Fri 15 Nov 2013 4:26PM

Others, like pidgin, are still lagging behind. WebRTC doesn’t allow unencrypted media traffic at all.

@philipphancke: What kind of encryption is mandatory? I guess telepathy based clients (KDE Telepathy, SailfishOS XMPP client and so on) won’t be able to interoperate with WebRTC based JavaScript XMPP/Jingle client as well? Telepathy still lacks end-to-end encryption support.

JB

JC Brand Mon 18 Nov 2013 5:33PM

Hi every1

I'm the author of https://conversejs.org which @guillaumelakano mentions two posts above me.

I think converse.js would be a good fit for Diaspora and one of my goals is to make it easy for people to integrate it into distributed FOSS projects such as this one.

I'd be happy to answer any questions or help out how I can if you guys decide to test-drive or use converse.js.

For the privacy minded, I recently added Off-the-record encryption support. [0]

I intend to add support for video chat, most likely using @philipphancke's strophe.jingle[1] library.

If anyone is interested in contributing, please let me know. Currently I'm working on this alone and in my free time.

[0] https://opkode.com/media/blog/2013/11/11/conversejs-otr-support/
[1] https://github.com/ESTOS/strophe.jingle

GL

Guillaume Lakano Mon 18 Nov 2013 6:22PM

@jcbrand It's good to see you on this topic ! :-)

If you include strophe.jingle of @philipphancke into your converse.js, it's will be perfect to add the XMPP/video chat on Diaspora !

GL

Guillaume Lakano Mon 18 Nov 2013 6:24PM

About the XMPP chat for Diaspora, we need to discuss about how to deploy it. Do we request to each pod to install an XMPP server ?

I think this could be usefull if D* users have a default account set on the pod AND have the possibility to change it to set their own XMPP account

G

goob Mon 18 Nov 2013 6:35PM

It's good to see you, JC, and thanks for taking the time to join this discussion.

Guillaume, on your last point, perhaps as you say users on pods which join the chat service have a default chat account, ideally set to be the same as their Diaspora ID (username@podname.tld). There is then a setting in user preferences 'Use an existing XMPP account'. If the user enters an existing account name in here, they can use that account in Diaspora, and any calls to their Diaspora ID get forwarded to this account, so someone can contact them via their Diaspora ID even if they don't know their XMPP account name. Does that sound like a good idea?

JB

JC Brand Tue 19 Nov 2013 9:49AM

Thanks @guillaumelakano and @goob

@guillaumelakano I was playing with strophe.jingle about a month or so ago and noticed that it is tested with Prosody XMPP server and doesn't yet work with Ejabberd (which is what I'm using ATM). I'm sure Ejabberd and other servers will eventually catch up, but this is something to keep in mind. This is bleeding edge stuff and broad support among browsers (for WebRTC) and XMPP servers is still lacking.

About deploying XMPP servers, I think we could perhaps do this organically. Can we not let pod owners decide for themselves whether they want to install an XMPP server or not?

There are public XMPP servers available (like jabber.org) and many people and companies host their own XMPP servers already. So certain people might want to re-use those accounts.

I can think of mainly two ways of adding XMPP support (both don't require an XMPP server per pod).

  • Tight coupling of XMPP credentials and Diaspora user account
  • Loose coupling of XMPP credentials and Diaspora user account.

Starting with tight coupling:

The important thing is that the user give their Jabber credentials to the Diaspora pod, and the pod then keeps a mapping of the user's Diaspora ID and Jabber ID and manages the users chat rosters.

If you "friend" someone, you'll automatically be able to chat with them (i.e. they'll be on your XMPP contacts roster) and there would be no need to add them again via Converse.js just to chat.

So, when you accept someone's friend request, the pod looks up that person's XMPP username and sends an XMPP contact subscription (on your behalf) to that persons. That way the new "friend" automatically also becomes a chat buddy.

This will however require server-side code that can talk XMPP and therefore take a lot of time to implement nicely.

And then loose coupling:

To quickly add XMPP functionality, you could let users manage their chat buddies via Converse.js. In this case, the pod doesn't need any new code, you don't necessarily have a 1-to-1 mapping between Diaspora users and chat users and when "friending" someone they don't automatically become a chat buddy.

It's not so tightly integrated, which sucks a bit, but it can be implemented relatively quickly (and the other functionality can then still be added in the meantime).

GL

Guillaume Lakano Tue 19 Nov 2013 10:39AM

@jcbrand About the choice for the admin to install or not an XMPP server, I think it's a "wrong good idea", because if only few pods install an XMPP server, we need a fallback (for example, accounts created on the default diaspora XMPP server) and this mean "centralized" ... it's never a good idea to centralize the majority of all accounts on one server.

Or... we can keep it as an option for the pod admin, but we really need to insist about the necessary to install it on their server...

About the way you describe for adding contact, to resume, when the web client D* receive the request :
- We search if this XMPP account is attached to a D* user.

  • No:
    We ask the question and precise it's not attached to a D* account (may be an option "ignore external" accounts could be added)

  • Yes :

    if I follow him, we automatically accept him, to permit to chat directly.
    If I don't follow him, we ask to the D* user if we accept his request with the D* name + link of this member.

M

Maciek Łoziński Tue 19 Nov 2013 9:45PM

Is this possible to make XMPP server a part of every pod and share contacts, domain and usernames between pod and XMPP server automatically? Is this reasonable? This could make some things easier for pod admins, which is important because we want to have thousands small pods rather than few mega-mods if I recall properly.

S

Shmerl Wed 20 Nov 2013 3:35AM

If you review the discussion above, you'll see that general consensus was about some kind of loose coupling of D* account / server with XMPP account / server. The main reason for that is flexibility and also the fact that some pods run on Heroku which doesn't allow running any other servers (like Ejabberd or the like). Therefore it should be up to podmins to decide whether to deploy XMPP server together with their pod or not.

M

Maciek Łoziński Wed 20 Nov 2013 11:24AM

But I don't mean server like ejabberd which is a separate application, rather my idea is to write XMPP server in ruby as a part of diaspora pod application. What about that? Would heroku allow app to listen on many ports?

S

Shmerl Wed 20 Nov 2013 3:51PM

I don't think you want to write your own XMPP server (let alone in Ruby) and keep it feature rich, unless you have time and resources for that. It's not a small feat.

S

Shmerl Wed 20 Nov 2013 3:53PM

Allowing using any external XMPP server (which can run on the same host when possible) looks like the most practical solution.

GL

Guillaume Lakano Wed 20 Nov 2013 4:01PM

@macieklozinski Well, your idea is interesting, but if we start to code an XMPP server in Ruby, this will take a lots of time.

@shmerl I think we doesn't have to take decision dependant of what Heroku propose. Of course, I understood some of podmins uses Heroku, but all podmins are not under Heroky fortunately ;-)

The idea of Maciek Łoziński to include an XMPP Server in Ruby is may be a way to discuss. I have found Vines ( http://www.getvines.org/ ), an XMPP server in Ruby.

@shmerl If we add a ruby XMPP server inside the code, do you known if it's help Heroku users ? Could they launch Diaspora AND Vines at the same time on their Heroku ?

If yes, we need to see with @philipphancke and @jcbrand if the Vines server is enough to support all features required for the chat / video.

This could be set as a default server but with the possibility to uses another server ( ejabberd, prosody ...)

S

Shmerl Wed 20 Nov 2013 4:05PM

I'm not familiar with Heroku, just know it's limited to Ruby on Rails and it's not a general purpose hosting, so let others comment on that. Writing Diaspora own custom XMPP server looks like a waste of resources to me, but of course anyone is free to do that if they want to.

GL

Guillaume Lakano Wed 20 Nov 2013 5:36PM

@shmerl The idea to create a custom XMPP seems a waste of ressources for me too, but it's why I propose to include an XMPP server coded in Ruby, open-source, Vines ( http://www.getvines.org/ )

S

Shmerl Wed 20 Nov 2013 5:53PM

It's not a question of what language the server is in, it's the question of what Heroku allows you to host. I think Ruby on Rails is not conductive for XMPP servers.

G

goob Wed 20 Nov 2013 7:05PM

There has already been a vote about the issue of Heroku, and the decision was that we should not limit our ourselves to finding a solution which works with (free) Heroku. See https://www.loomio.org/discussions/3678?proposal=2643

Any solution needs to be compatible with a network which contains pods hosted on free Heroku, but the chat solution does not have to be one which can be installed and run on a free Heroku account.

F

Flaburgan Wed 20 Nov 2013 8:30PM

@guillaumelakano @sakwe already proposed vines a few days (weeks) ago. We are looping in our discussion guys ^

So to summarize, as goob said:

  • Heroku support is better but not necessary
  • What we want first is a chat. Video would be nice, but a simple XMPP chat would already be good.

Now if someone (@sakwe ?) could code a simple proof of concept using vines to see if it's light or not, and if it works on heroku, that would be nice.

D

diasp_eu Wed 20 Nov 2013 8:48PM

@Goob Diaspora on free Heroku account with the current federation system will not work, because of the replication of posts the db with max. 10k rows is the end. Otherwise the user needs to pay. We should not focus on Heroku.

Vines sounds good - first we need a proof of concept. Let's start coding :-)

G

goob Wed 20 Nov 2013 8:52PM

@diaspeu

We should not focus on Heroku.

That's what I'm saying!

JR

Jason Robinson Wed 20 Nov 2013 9:32PM

It would be odd for a free software project to focus on making sure hosting works with one commercial provider.. ;)

GL

Guillaume Lakano Thu 21 Nov 2013 4:29PM

So, if you do not need to worry about Heroku, we can simply ask to podmins to install an XMPP server of their choice, it's simple :)

1) So, we use the same D* ID for the XMPP ?
I mean, my D* ID is guillaumelakano@diasp.org then, this will be also my XMPP account, isn't it ?

2) if a podmin doesn't want or can't install an XMPP server, do we plan an fallback ?
I propose to make this simple, and to not make fallback, because fallback mean a centralized XMPP server (and it's never a good idea)
So, if users want chat services, they need to change to a pod with chat support.

3) We also need to discuss about the possibility to change pod. What happends to our previous XMPP account.

G

goob Thu 21 Nov 2013 5:14PM

I think it would be enough to implement a solution which allows pods which have an associated XMPP server to communicate with 'chat' with other pods which have an associated XMPP server. Not every pod in the D* network would have to participate in this chat solution.

Users who wanted to participate in chat could move to a pod which provided the facility. Podmins who didn't have the resources to run an XMPP server needn't be under pressure to try to install one in order to continue their pod within the D* network.

S

Sakwe Fri 22 Nov 2013 7:25AM

Yes, I hope finding some time to work on a little code I have proposed with Vines. I confirm that Vines works as an independant XMPP server. I have installed this from the Ruby gem. BUT I'm not a Ruby programmer so, I have to learn a little bit about it...

The code I propose to write is an adapted code of this gem (https://github.com/negativecode/vines-sql) that enable Vines XMPP server to use a SQL database instead of the master storage managment.

I think that it could be enough to make the Vines XMPP server totally integrated to the D* database.

So we could imagine that a Podmin would have to install the Vines server and our "Vines-D*-stograge-adapter" gem with the bundle and all would be ok for the server part of the chat capability.

BUT with this solution, the XMPP server is not an external service and it become a part of the node Pod (the D* node become a XMPP node).

Personnaly, I prefer this because final users don't have to create another XMPP acocunt to chat on D*. Their D* account is their XMPP account. Their D* aspects are their XMPP groups, their D* contacts are their XMMP contacts.

The SSL certificate is the same for both services...

PS : I have a question if someone can answer me... I'm making tests on Jappix Mini integration with a "pre-login" (login/pass from a database). Is there another way to auto-login Jappix Mini than with the "launchMini()" function ? Or if no other way, is it possible to crypt the password we ha to feed in the function call ?

JR

Jason Robinson Fri 22 Nov 2013 9:03AM

Personally I think we should make pods lighter, not heavier. It's already quite resource heavy and people want to run pods on light stuff like Pi's or even routers. We're not going in the right direction if we bundle more functionality in.

But I guess with proper planning components could be opt-in and thus keep the core light and the extra weight as optional.

S

Sakwe Fri 22 Nov 2013 10:21AM

Sure it is an important question. I can say that the XMPP service is very light so I don't consider it as a real overload for the machine.

As Vines is a Ruby gem, a podmin that can install D* on a server can install the XMPP server. It only need some openned ports on the server.

I think that the best solution for D* users is to have automatically access to the chat capabilities with the same ID for both services.

Asking an user to add separeted accound with no link with his D* datas (essentially the id) is not a real integrated chat. Most of users will not unedersand this, only "power users" will use it I think... No?

JR

Jason Robinson Fri 22 Nov 2013 10:32AM

Sure it needs to be integrated, agree totally there :) But that can also be done via provisioning. But didn't really want to interfere on the XMPP discussion, no knowledge here, just saying it's not necessarily always a good idea to bundle everything together into one component :)

D

diasp_eu Fri 22 Nov 2013 1:24PM

We should learn from our failures :-) 28 changed files with 12,708 additions and 3 deletions without any result:

https://github.com/diaspora/diaspora/pull/1750

In Vittorios approach we used ejabberd. Vines sounds better because of the seamless diaspora integration.

G

goob Fri 22 Nov 2013 4:03PM

I think the thing to learn from that project is 'don't develop anything for a project whose core developers are making enormous changes to the code base without communicating this'. This is no longer the case! And once more modularity is built in, changes to the core code base will be even less of a concern.

S

Shmerl Fri 22 Nov 2013 4:23PM

I think D* shouldn't mandate the XMPP server. It can be Vines, Ejabberd, Openfire - whatever the podmin likes more. D* should be decoupled from the XMPP server. If podmin can't install it (Heroku, etc.) - then that pod won't have chat feature. Alternatively, the UI can allow connecting to external (hosted elsewhere) XMPP server.

S

Shmerl Fri 22 Nov 2013 4:30PM

Decoupling won't tie D* to the limitations of the chosen server, and if some XMPP server will get ahead in features or development, podmins will be able to easily switch. Hard tying D* to one server will make that impossible.

S

Sakwe Fri 22 Nov 2013 9:35PM

If we want to have an integrated chat functional that offer the possibility to be decoupled from the pod, it mean that IDs/passwords are not the sames on both services. It means that databases for both services will not be the sames. It means that the end-user have to understand the difference between D* and chat in D*.

In this way, if we want an "integrated" service, we have to code functions that will synchronise (in the best way) the D* contacts and aspects with the XMPP contacts and groups. I think it can be done by BOSH communation.

So, this solution could exits but it means I think a lot of new code in the D* core to send, receive and confirm BOSH messages with the xmpp service.

Example: A new "aspect" is created, we have to send a BOSH request to create the same "group" in the xmpp account. (same for contacts, add, delete, move, ...)

I think that we loose a part of Pods federation with this solution too...

Example : If "Xuser@Ypod.none" want to communicate with "Aperson@Bpod.none" on the D* network, it's ok, the federation is done. But if "Xuser@Ypod.none" have an xmpp account "Xuser280@xmpp.none" and "Aperson@Bpod.none" have "A69person@bxmmp.none" as xmpp ID...

How does the federation could help us without a new hard functionallity in D* and a global update for all Pods?

And still the same question. How many users will undestand that they have a D* id/pass for D* but a different xmpp id/pass to chat ?

S

Sakwe Fri 22 Nov 2013 10:59PM

I add this reflexion... I think that decentralisation have limits : the user. It's not "user friendly" to decentralise a user.

EK

Emmanouel Kapernaros Fri 22 Nov 2013 11:21PM

I believe there is no need for a user to use some preexistent xmpp id. If he has already an xmpp id, he probably uses it with pidgin or some other app. He can use his new diaspora xmpp id with pidgin too.

So I think it is better not to have the option to import an external xmpp id into diaspsora. Every diaspora id will be automatically an xmpp id too.

S

Shmerl Sat 23 Nov 2013 11:24PM

Coupling of the server with D* has more problems than benefits which a unified ID gives. You can make enough integration with a flexible XMPP server decoupling.

Example:

  1. User creates an account on a D* pod which has associated XMPP server configured. Under the hood, account is created on that XMPP server which matches D* handle, and with the same password. When user changes D* password, XMPP password is changed accordingly.
  2. User is provided an option in UI, to bind these IDs, or to unbind them (by default - they are bound). When bound, D* will use the matching JID, when unbound, user can set explicit XMPP id on any other server.

This allows several things:

  1. No need to require Vines only. Any XMPP server will do, as far as there is a unified way of creating a user there and setting / changing passwords.

  2. It gives an option for "power" users to use other XMPP handles besides the default one.

S

Shmerl Sat 23 Nov 2013 11:27PM

Handling authentication for presence and contacts can work similarly. Not everyone I add on D* in my aspects should be authenticated for my XMPP presence, and the other way around too. User should be given a choice, since those aren’t necessarily always matching 1:1.

SM

Seth Martin Sun 24 Nov 2013 7:11AM

I believe that it would be more user friendly when users are able to use their existing XMPP accounts from whatever server they want. Many of these users probably already have accounts with facebook and google to add in their Pidgin and Jitsi clients. Avoid the confusion of having two additional unchained XMPP accounts and having to ask all their contacts to add yet another ID for them.

It wouldn't surprise me if most podmins are already running an XMPP server and would like to continue only running one.

GL

Guillaume Lakano Sun 24 Nov 2013 2:31PM

As @emmanouelkapernaro said, it's surely better to not uses an existant XMPP account for users.

I propose to make it simple to report the feature of using an external account for later.

Like that, it's better to concentrate only on 1 account created on the fly on the pod, with the same D* ID

F

Flaburgan Mon 25 Nov 2013 10:14AM

Yeah, I also think that using the existing handle as the xmpp id is the simplest and less confusing way to go. @sakwe I encourage you to work on a proof of concept and ping DenSchub, MrZYX or Raven24 on #diaspora-dev if you need help about Ruby.

PP

Pirate Praveen Tue 3 Dec 2013 9:36AM

I think we should let @sakwe continue with Vines integration. If anyone feels strongly the XMPP server should be decoupled, they can implement that too. It can be an option for the podmin to choose. The important part here is to get chat working as soon as possible. If someone is working on code, lets not block them with tougher requirements.

S

Sakwe Wed 4 Dec 2013 3:03AM

As I see the "Vines solution" seems credible, it motivates me to write this code. Unfortunately, I am still very busy this December to devote myself. In January, I will work on it.

D

diasp_eu Fri 6 Dec 2013 7:09AM

What are the tasks for the Vines integration?

S

Sakwe Sat 7 Dec 2013 1:14AM

@diasp_eu The first code to write is an adapted code of this gem (https://github.com/negativecode/vines-sql) that enable Vines XMPP server to use a SQL database instead of the master storage managment.

I think that it could be enough to replace Vines data queries into its local database to queries into D* database.

AA

Alberto Aru Sat 11 Jan 2014 7:10PM

Happy new year to everybody!

@sakwe thank you for helping to the XMPP integration, when could you start working on it?

I comment because I don't want we forget this excellent post on Loomio with the christmas holidays.

S

Sakwe Wed 22 Jan 2014 3:24PM

Unfortunatly, I need to work on another project for the moment and it take all my time. But I don't forget this functionnality and I'll have time soon to work on it.

AA

Alberto Aru Thu 23 Jan 2014 11:43PM

Ok, thank you anyway and good luck!! :)

V

Vostok Wed 12 Feb 2014 11:32AM

Hi guys.
I image this is the right place to know about, so how is going the chat development?
Is it going to be XMPP based?
I know we need to be patient about that but I asked because there are many people asking me all time and I have no answer to give them. Tks

SM

Seth Martin Wed 12 Feb 2014 8:00PM

I certainly hope we don't stray away from the XMPP plan, it seems that is what most of us want currently and would be ideal.

We still need a developer that has time for this.

V

Vostok Wed 12 Feb 2014 11:19PM

Cool, the XMPP is indeed an amazing choice.
I'm gonna try to invite some developers I know, let's see what happens. Unfortunately I can't help much on dev.
Thank you @sethmartin

DT

David Thompson Fri 14 Feb 2014 3:07AM

Hi, I have done some work in the past year to integrate a sitewide XMPP chat system for the company that I work for. The application itself uses rails, for an XMPP server we chose Openfire plus the Punjab BOSH server, and we wrote a JavaScript client on top of Strophe. While I am by no means an expert on all of the intricacies of XMPP, I feel that I could offer some assistance. I believe we've successfully served 10k+ users (being conservative but we stress tested a lot higher than that) online at once for anyone concerned with performance.

I agree with some of the previous comments that the XMPP server shouldn't matter and could be left up to the podmins. A JavaScript XMPP chat client can be written to work with any BOSH server. For those that don't know, BOSH wraps XMPP in HTTP.

I see that the other alternative is Vines, which I haven't used so I don't have an opinion. If that is the better choice, by all means go for it.

WDYT?

DT

David Thompson Fri 14 Feb 2014 5:10PM

As an addendum to my previous post, figuring out how to authenticate users on a standalone XMPP is an interesting problem that would likely require extensions to each server that a podmin wanted to use.

JR

Jason Robinson Sun 16 Feb 2014 11:58AM

@davidthompson awesome that you are offering your help here - a lot of users really want chat but we just lack the hands to do that - so help would be appreciated!

There has been a lot of talk about different options but really no discussions. I think the way forward would be to make a concrete suggestion of technical detail (maybe in our proposals page in the wiki, or just here briefly) - and then we either accept it, modify it or reject it, by voting if necessary. In the end, the technical solution should strongly be influenced by the person who is actually willing to do the work ;)

JR

Jason Robinson Sun 16 Feb 2014 11:59AM

Sorry, "no discussions" = I mean no concrete full technical proposals that could be accepted :) There has been a lot of discussion of course.

TS

Trolli Schmittlauch Sun 16 Feb 2014 6:08PM

Relying on XMPP as the basis of a Diaspora* chat is a great idea, but we have to consider that plain XMPP isn't really usable on mobile devices. So when thinking about which server to use or which featureset to implement please consider that it should work great with mobile devices.

This article describes why the current state of most XMPP clients and servers sucks for mobile use: http://fnanp.in-ulm.de/blog/2014/01/16/01-woes.html

We have the chance to do things right from the beginning!

JR

Jason Robinson Sun 16 Feb 2014 7:25PM

I have Xabber on my phone and it works great - though I just idle there since I don't really know anyone through there lol. Would be great to be able to chat with d* guys with a mobile client.

TS

Trolli Schmittlauch Sun 16 Feb 2014 7:51PM

@jasonrobinson according to the blogpost Xabber is one of the clients which don't support the important XEP-0198 which sometimes causes a silent loss of messages. And this is what I mean

S

StarBlessed Sun 16 Feb 2014 8:04PM

I use Jitsi on my mobile, it works just fine here. Including voice and video (read Jingle).

F

Flaburgan Mon 17 Feb 2014 11:43AM

@davidthompson maybe you can see with @dennisschubert and @sakwe who proposed their help too.

SM

Seth Martin Tue 18 Feb 2014 6:11AM

I wouldn't want any XMPP solution that didn't permit my using Prosody for the server. Prosody has an XEP-0198 extension which is necessary to be reliable for mobile users as @trollischmittlauch points out. I wouldn't want to prevent any podmin from choosing their own though.

As for mobile clients, ChatSecure is the real winner at this moment. I was using Xabber which was very nice but after discovering that XEP-0198 support was an absolute must for mobile, ChatSecure was the obvious choice. The article that @trollischmittlauch links to above, claims this client to have connection issues but I haven't experienced this on my device.

DU

[deactivated account] Wed 19 Feb 2014 4:56AM

I'll say again that you can use a server component with any XMPP server out there. If the server component implements XEP-0100 (easy to implement, see Libertree code) then a user can connect any JID (even on remote servers) with the Diaspora account. If you don't like that an existing JID is required to use it with a regular XMPP client you can transparently create an account on a local XMPP server (even with the exact same name as the user handle), because there's a XEP for that.

Integrating a full XMPP server in D* is against the spirit of XMPP and is a terrible idea from a software development perspective.

What to use on the frontend-side is a different matter.

DU

[deactivated account] Wed 19 Feb 2014 5:01AM

@trollischmittlauch "silent loss of messages" is always a possibility with message stanzas. Iq stanzas, on the other hand, have to be acknowledged by the receiver and are retried if there is no response.

ASB

Alexander Salas Bastidas Thu 6 Mar 2014 6:53PM

I can make the implementation for Pidgin, Instantbird and Thunderbird with libpurple protocol plugins.
We are waiting for deploy the service.

JR

Jason Robinson Thu 6 Mar 2014 7:59PM

@alexandersalas what do you mean by "make the implementation" and "waiting to deploy the service"?

ASB

Alexander Salas Bastidas Fri 7 Mar 2014 12:50AM

@jason-robinson Pidgin need plugins [1] for new protocols (example: prpl-diaspora) with libpurple [2], i can make the implementation of this new protocol for Diaspora, i make in the past another addons for Instantbird for example [3]

[1] https://developer.pidgin.im/wiki/ThirdPartyPlugins#AdditionalProtocols
[2] https://developer.pidgin.im/wiki/WhatIsLibpurple
[3] https://addons.instantbird.org/en-US/instantbird/addon/358

RF

Rasmus Fuhse Fri 7 Mar 2014 8:15AM

Thanx for your offer!

We definitely don't want to invent a new format for chatting. We will be adapting XMPP and so pidgin will be able to connect with diaspora out of the box. So I'm not sure if there is anything to be done for pidgin at all. Maybe extending the menu for a diaspora user to easily access the profile of that user from pidgin or something like that?

DU

[deactivated account] Fri 7 Mar 2014 3:44PM

@rasmusfuhse If Diaspora behaves as a regular gateway service the client only needs to implement the gateway workflow to allow for chat with Diaspora users. Pidgin and Psi are just two of many XMPP clients supporting service discovery and gateway interactions.

D

diasp_eu Wed 12 Mar 2014 7:40AM

After 11 month of discussion we should vote for a technology and go for it.

V

Vostok Thu 13 Mar 2014 12:13AM

In this meanwhile, does anybody know what happened to this project below?
http://blog.ninux.org/2011/08/30/gsoc-2011-videochat-for-diaspora/

RF

Rasmus Fuhse Thu 13 Mar 2014 12:16PM

The author of this blog-article is @vittoriocuculo who also brought in some comments here. Back at the time there were automated tests missing, so his pull-request was rejected. Right now his code is unmergable and Vittorio doen't have so much time anymore. But his work can be used to reimplement the chat and his experience will be helpful.

BC

Balasankar C Mon 17 Mar 2014 2:11PM

So, we fixed XMPP for chat, or should a new decision proposal to be opened? And, if it has been fixed, what is the coding status?
Sorry for being impatient (and that too without help in the coding side, but when it comes to Diaspora, I am a mere user and evangelist, and never a coder. Don't know Ruby or Rails. :( ), but when I speak to people about D*, they ask me about chat and I really don't have anything to say.

F

Flaburgan Thu 20 Mar 2014 12:46PM

@balasankarchelamat we agree the way it should be done, but nobody started to work.

V

Vostok Thu 20 Mar 2014 9:37PM

Hey guys, maybe some of you already know about that but there's an XMPP API code called Jappix Mini.
I'm not a developer and I'm also not sure if this could help somehow for a start, but the javascript code is in the link below, just select option 4 (Installation):
https://mini.jappix.com/get#

@rasmusfuhse , thanks for the answer.

BK

Brad Koehn Fri 21 Mar 2014 3:17PM

I've started messing around with the jquery xmpp plugin, which is BOSH based. The first thing I ran into was the need to conflate the user's Diaspora contacts' JIDs into the user's XMPP account's roster (see https://diaspora.koehn.com/posts/148942). It looks like you're thinking that the pod should have access to the user's XMPP account, which is pretty much the only way for this to work. There's a lot of server-side coordination to keep the two accounts in sync, which is complicated somewhat if the user logs in with another XMPP client and starts to add/remove JIDs themselves.

Assuming we can solve this problem, the next problem I see is that if you have a widget on screen into which you're chatting, the widget will disappear every time you switch pages. If the widget is doing the XMPP/BOSH stuff itself (which probably makes the most sense) then the user will lose their chat state (history, partially-composed IMs) every time they switch pages as they look at individual posts, change aspects, etc. since all of those involve changing pages.

We'll need to make Diaspora into a one page application to keep the widget visible as the user navigates the pod, which is what Facebook and Google+ do. That's a significant chunk of work, I'd imagine. You might be able to hack something together with frames, but everyone hates frames for good reason. A last option would be to have a separate window for chat, but that has negative implications for mobile.

BK

Brad Koehn Fri 21 Mar 2014 3:44PM

BTW My Jabber ID is bkoehn@koehn.com. :-)

BK

Brad Koehn Fri 21 Mar 2014 7:15PM

A diaspora user suggested checking out converse.js; it does most everything we'd need.

JR

Jason Robinson Fri 21 Mar 2014 7:57PM

Awesome @bradkoehn :)

Personally I'd be happy initially to just integrate XMPP provisioning only to start with. This way podmins can run a server themselves and auto-provision accounts for their users. For contacts, xmpp ID would show in the profile.

Then second step - integrate chat into UI.

BK

Brad Koehn Sat 22 Mar 2014 2:35AM

Any suggestions on how to do this? I imagine it's partly a matter of configuration: the domain of the pod should match that of the XMPP server and an admin account must exist on the XMPP server which the pod can be configured to use to create XMPP users with the same password used in Diaspora. A secondary requirement is that the XMPP server should support BOSH and have the appropriate SRV and TXT records to make it easy to communicate with.

Once that's done we need to add the user's XMPP id to their user record and begin transmitting it to other pods. I think that that meets the requirements of Jason's first step.

What else needs to be done? And how do I set up an RoR IDE?

DU

[deactivated account] Sun 23 Mar 2014 8:27AM

@bradkoehn Have you evaluated the XMPP server component approach yet? Chat can be handled very easily within a server component as you are not restricted by XMPP stanzas. That's what we do in Libertree --- there is no additional requirement on the XMPP server side. Everything is handled by the component, allowing for a minimalist chat implementation. No BOSH needed. The client doesn't have to speak XMPP at all, that's the job of the component. This also makes it possible to have chat without XMPP (at a small performance cost).

BK

Brad Koehn Sun 23 Mar 2014 2:03PM

That's certainly another option. It would seem to put forth some new requirements for developers (mostly that a chat component be available for Ruby to act as a gateway) while eliminating some complexity for podmins (setting up an XMPP service is onerous, as I just discovered getting ejabberd running on my host). It's certainly worth considering.

BK

Brad Koehn Sun 23 Mar 2014 2:19PM

@rekado How did you handle the problem of switching pages in Libertree? Or did you? converse.js does its best (I've been trying the roundcube plugin), but the user still loses information when switching pages. Migrating Diaspora to a one page app à la Google+ or Facebook would be a massive undertaking, but it's required to get chat working correctly, especially if one adds voice/video.

DU

[deactivated account] Sun 23 Mar 2014 11:08PM

setting up an XMPP service is onerous

My experience with prosody has been the opposite. It was less work than setting up an HTTP server.

DU

[deactivated account] Sun 23 Mar 2014 11:22PM

How did you handle the problem of switching pages in Libertree?

I don't know why you think that would be a problem at all. Every chat message passes through the XMPP component by design. Hence it can be persisted and replayed to the user on demand. We simply store recent chat messages in the database.

We don't use converse.js because it's not needed. We only need to speak XMPP in communications between servers, not between backend and frontend. In Libertree chat messages received by the backend are stored in the database and pushed through to the frontend via websockets. Really simple. We are not dealing with stanzas in the frontend, because that is not required to make chat work. We don't need a full XMPP client in the frontend thanks to the privileged position that an XMPP server component has --- it can author stanzas and the XMPP server transmits them. We have no need for any of the usual tasks of an XMPP client because we only need to talk to the XMPP server component using whatever means makes sense (in our case: through the database and websockets).

A chat message sent off in the frontend has to go through the XMPP server component as well. We never lose conversation state and do not need to implement all that in JavaScript either. You are welcome to copy chunks of our code if you find that helpful. See the libertree-backend-rb repo.

JH

Jonne Haß Mon 24 Mar 2014 9:16AM

I guess the real question we need to ask with this approach is, whether it scales to a few thousand concurrent users that some Diaspora pods have.

BK

Brad Koehn Mon 24 Mar 2014 6:49PM

@rekado Thanks; I hadn't thought it through that far. So you must be sending keystrokes up to the server, and thus maintain a lot of state server-side (e.g., who you're chatting with, messages you've entered but not sent) that I was thinking would stay in the client. Because by the time you've built all the state into the server, you may as well handle the message transport too and forgo XMPP altogether, or have a gateway that allows XMPP through.

The TCP socket limitation is certainly an issue; we'd have to consider the limitations this would prevent in terms of deployment. Some advantages of BOSH are that it doesn't open a connection to the web server, and the XMPP connections aren't TCP connections kept open all the way to the browser. That makes scale much easier to manage; the downside being that there are connections from the browser going to multiple servers and the loss of state on page loads.

I'm starting to grok why this feature hasn't been addressed sooner.

DU

[deactivated account] Mon 24 Mar 2014 11:43PM

@bradkoehn You misunderstand. The browser does not open TCP connections to remotes. TCP connections are managed by the XMPP server. It establishes connections to remote servers as needed. The component only maintains one stream: to the XMPP server.

I mentioned websockets as one way to push messages in realtime to the browsers. This means one websocket connection for every active user. Websockets are no requirement but they have less overhead than periodic HTTP requests that would be needed for polling (or BOSH for that matter, because then the client would have to speak proper XMPP, which is a silly requirement).

The state you are talking about is transient; there is no need to "build" any state "into the server". The client sends stuff to the component which then asks the XMPP server to handle the rest. I don't understand where you see complexity in this setup.

DU

[deactivated account] Mon 24 Mar 2014 11:49PM

@jonnehass The only thing that may not scale too well is the need for a communication channel between browser and Diaspora server. There will be one websocket connection per browser. This is less overhead than periodically establishing a HTTP connection to poll for changes. Any production websocket server should be able to handle a high number of concurrent connections, though. This remains to be seen, but is not critical as the use of websockets is entirely optional in this scenario.

JR

Jason Robinson Tue 25 Mar 2014 8:31AM

alternative to "just" websockets is something that falls back to older browsers, like socket.io

D

diasp_eu Wed 14 May 2014 5:56AM

Nice Diaspora WebRTC chat feature https://diasp.de/posts/1384079

JR

Jason Robinson Fri 16 May 2014 8:15PM

Interesting :)

LM

Lukas Matt Wed 28 May 2014 9:26AM

@diaspeu I played with that, but I am not sure whether I should continue developing

cause of the Mozilla Loop project.

What is the opinion about that?

Should we use such alternative and think about only implementing the "address-book"?

( If webrtc is the thing we want ;-) )

Regards,
Lukas

A

Anahuac Wed 28 May 2014 10:47AM

After reading all comments here, basically we all agree that Chat is a very important feature and it must to be implemented.

Why we don't just make it as simple as possible in it first version?

As others my opinion is that Diaspora got a XMPP client from within that allow users to setup any XMPP account.

In diasporabr.com.br POD we already setup a Prosody XMPP server integrated with Diaspora database to use it's accounts and passwords.

I have to say that Prosody is a very nice piece of software.

If each POD install it's own XMPP server and Diaspora got a XMPP client the main issue - get a chat in Diaspora - is solved.

I agree that WebRTC is the most beautiful and featured option, but this will probably take quite a time to implement.

As you can see this topic has been discussed for more that two years now. Lets implement it as easy as possible.

I vote for a simple XMPP Client within Diaspora.

A

Anahuac Fri 30 May 2014 5:22PM

Hey,

This is my first time in Loomio so I'm curious about the vote feature.
How does it work? In some point the voting button will be enabled by some one? It is intelligent enough to read each message and figure it out?

Some can give me some insights?

Thanks

G

goob Fri 30 May 2014 5:38PM

Hi Anahuac, and welcome to Loomio.

The voting function is only available when there is an open proposal. On this discussion there is only a closed proposal, which ended a year ago.

If you look on discussions with open proposals (you will see them at the top of your discussion list, and with coloured pie charts next to them) you will see how you can vote on those proposals.

Anyone can start a proposal, but please read our wiki article on discussion and voting first.

Hope that helps.

A

Anahuac Fri 30 May 2014 8:00PM

Hi Goob,

I see... thanks for your answer.

So what we are discussing here are not under voting, so what are we doing?

Is this to get some insights to the subject?

I mean, I'm not a Ruby developer, so I can't help in the Chat development, but how kind of effort I need to do to see Chat implemented on Diaspora?

I'm in charge of the diasporabr.com.br POD and every single day I got a couple of people asking me about chat and personal pages.

I really love the idea of offering both to users.

Regards

G

goob Fri 30 May 2014 9:16PM

Well, the discussion about how to implement chat has taken place a long time ago (look at the previous pages on this thread). It now just needs people to volunteer to start working on it! Several people have offered, but always something has happened to stop them (other commitments, and so on).

Unfortunately, like you I'm not a coder so I can't help. If you know anyone who has those skills and might be willing to work on this, send them this way!

A

Anahuac Thu 5 Jun 2014 12:39PM

Well, I know that this is probably not the right place to ask, but... is there, anywhere, the API documentation?

V

Vostok Mon 16 Jun 2014 11:26AM

Hi Anahuac, good to see you here, welcome!
As I see you've successfully implemented XMPP on your pod, so even if we didn't decided here which service to use for the chat yet (XMPP or WebRTC or whatever), how about telling us your first impressions using Prosody as the XMPP base? I mean, you could tell us about the use of server's resources, the capacity to handle many concurrent users, the integration between the XMPP server and the Diaspora handles, and so on.
I have the very same issue here: many users are always asking me about a chat implementation and I'm wondering if I also implement a XMPP server until this decision is made by the community, and until the final solution is ready for the Diaspora pods by the code.

A

Anahuac Wed 18 Jun 2014 6:28PM

Hi Vostok,

So, my very first impressions of have Prosody as XMPP server are grate. This is a very light program and the server resources have not been consumed at all. But I have to say that no many accounts are using it, so I really can't say how the server will deal with thousands of users.

The main reason to most of the account don't use XMPP is that regular users don't want to install clients out from diaspora and this is why I would love to see a XMPP client integrated on Diaspora.

Abut the technical integration it was a little bit tricky. A had to touch a lot the SQL plugin file to allow prosody authenticate against Diaspora password table.
I have to write a howto and publish it on thw diasporafoudation.org wiki and I will do it soon. Problably next week, because I'm going out for a little vacation for the Saint John holiday in Brazil.

A

Anahuac Thu 26 Jun 2014 7:55PM

Finally I got some time to do the Prosody + Diaspora howto. It is here:

https://wiki.diasporafoundation.org/Integration/XMPP/Prosody

take a look at it and give me some feedback

G

goob Thu 26 Jun 2014 8:26PM

Hey Anahuac, thanks a lot for doing this. I hope you don't mind, I've just gone through and edited the English (not much to do, you speak it pretty well!) and added inline code markup where I thought this was appropriate.

Do check I've not added any errors or confusion: here's the diff.

F

Flaburgan Thu 26 Jun 2014 9:56PM

@anahuac it looks really nice!

The next step is to automatically add a contact in XMPP when someone adds it to his aspects.

Then we will have to add a chat in the interface of diaspora*, maybe with something like jappix mini?

A

Anahuac Thu 26 Jun 2014 10:02PM

@goob I do not mind! In fact I appreciate it very much. Thank you for the contribution :-)

@flaburgan This is the idea! Now developers need to figure how to implement a XMPP client inside Diaspora interface :-)

JR

Jason Robinson Fri 27 Jun 2014 7:35AM

Wow this is pretty awesome work @anahuac ! Will definitely give it a try on my pod.

Anyone want to work on integrating a chat client on the d* UI side? It should imho not be tied strongly to this Prosody solution though.

A

Anahuac Fri 27 Jun 2014 12:02PM

@jasonrobinson I'm glad you like it. This is how I can contribute to our beloved Diaspora :-)

As I said before, I'm very excited to see a chat client on the d* UI.

SM

Seth Martin Sat 28 Jun 2014 2:42AM

Oh @anahuac , thank you, thank you, thank you!

This is exactly what I was looking for.

I agree with @jasonrobinson that an integrated chat client should not be tied strongly to this Prosody solution though. However it wouldn't bother me at all since Prosody is my choice anyhow.

DU

[deactivated account] Tue 8 Jul 2014 8:34PM

A XMPP chat dev for D* has started bitween Lukas @Zauberstuhl and me.

The solution use Vines ruby gem for XMPP server and Jappix Mini as frontend. Lukas is going to do an official pull request in the next couple of days.

G

goob Tue 8 Jul 2014 8:46PM

@augier that sounds great!

DU

[deactivated account] Tue 8 Jul 2014 9:42PM

Which solution would be the best ? Prosody, or Vines ?

ST

Sean Tilley Tue 8 Jul 2014 11:36PM

@augier Vines looks really cool.

However, one thing that I don't understand is that some of the instructions imply the need for CouchDB, but it doesn't state that in all of the instructions. If anybody can confirm that we don't need CouchDB to make use of it for Diaspora, Vines would definitely get my vote.

M

matl Tue 8 Jul 2014 11:46PM

Prosody would be awesome.

V

Vostok Wed 9 Jul 2014 2:25AM

@anahuac thank you for publishing the howto on the wiki, nice work man!
I've implemented a XMPP server with Prosody on the pod and it's working pretty well so far.

@augier wonderful news!
The authentication between Diaspora SQL database and Prosody XMPP server is working fine by the auth script modified by Anahuac.
About Vines, I've seen it's similar to Prosody but it also allows the config of components (access to subdomains using passwords) and clustering of server instances (load-balancing using Redis) which I'm not sure if Prosody allows. Maybe Vines has more features than Prosody, need to be checked.

@seantilleycommunit I found this, not sure if helps:
"Vines can store user data in several databases out of the box: Apache CouchDB, Redis, MongoDB, any SQL database, or simply in the local file system."

A

Anahuac Wed 9 Jul 2014 2:34AM

@vostok Thanks! As I said before this is my small contribution, since I'm not a Ruby developer.

@augier wonderful news indeed!
My only concern is about performance. Did you, or anyone else, have some comparative between Vines and Prosody?

What makes me choose prosody was the fact that the jabber-br.org XMPP server migrated to it a couple of months ago exactly for a performance issue. The second reason was the fact that Prosody is developed in Lua that is a fast and very robust dev language.

DU

[deactivated account] Wed 9 Jul 2014 5:33AM

@anahuac : Nope, no comparative. We weren't aware about the existance of Prosody. But as the interface is totally separated from the server, we can change with only few modifications. But the question is interesting. I only followed the work started by Lukas to add Jappix Mini.

@seantilleycommunit : Some features need CouchDB, indeed, as seen on the website. But I didn't deepen the question further.

A

Anahuac Wed 9 Jul 2014 12:06PM

@augier What about do it in a way that you can choose any XMPP server? I know this mean more work, but I think this will be grate stuff to have since each POD could choose the best server for it's own needs.

I'm very excited to see this up and running. Keep doing what you are doing and let us know of yours progress.

JR

Jason Robinson Wed 9 Jul 2014 12:31PM

What about do it in a way that you can choose any XMPP server? I know this mean more work, but I think this will be grate stuff to have since each POD could choose the best server for it’s own needs.

It would be good if the XMPP server would be configurable by the podmin - so separating the chat UI/code and actual server parts. Reason - many podmins already run an XMPP server.

Awesome work guys - totally looking forward to this! You will be heroes if you finish this :)

DU

[deactivated account] Wed 9 Jul 2014 6:59PM

@jason-robinson : The UI is totally separated from the server. So yes, it is possible.

@anahuac : even though this is perfectly possible, let's make it work with one server for the moment ^
Then, when can write a standard JS interface to connect the UI and the server, and a ruby script to do the configuration, yes !

There's a lot of work in here ! :D

A

Anahuac Thu 10 Jul 2014 3:46AM

@augier it's perfectly ok by me :-) keep doing it!

JR

Jason Robinson Thu 10 Jul 2014 6:52AM

Sure, small steps, then iterate :)

DU

[deactivated account] Fri 11 Jul 2014 5:38AM

A first version is finished now. See demo here : https://www.youtube.com/watch?v=eLN0PK7qmXw

JR

Jason Robinson Fri 11 Jul 2014 8:30AM

That is super awesome :) Do you guys have a pull request to submit so that review could start? Doesn't have to be in any way finished for merging.

BC

Balasankar C Fri 11 Jul 2014 8:30AM

I created a test account in sechat. But I am seeing "Error" in the chatbox. See attachment. Problem in my side?

LM

Lukas Matt Fri 11 Jul 2014 9:14AM

@balasankarchelamat your browser is blocking the http-bind request cause it is currently non-ssl.

I am fixing that today! It should work with chrome and firefox (if you disable the security check).

Cheers

G

goob Fri 11 Jul 2014 10:33AM

@lukasmatt Do you have a route by which people can provide bug reports? Here isn't really the right place, and I've started receiving bug reports to Diaspora HQ. If you've got a GitHub repo that would be perfect - alternatively is there a Diaspora account at which you're happy to receive them?

LM

Lukas Matt Fri 11 Jul 2014 10:38AM

Feel free to report to:
https://github.com/zauberstuhl/diaspora

@jasonrobinson my next update includes some configuration changes which will give the podmin the ability to disable the chat or the internal server. That means as long the podmin knows what he is doing he can have his own xmpp server.. that is the awesome thing with XMPP it's a standard... no need for big changes :)

DU

[deactivated account] Fri 11 Jul 2014 12:25PM

@lukasmatt : @goob is right : we should make a D* page for help and bug submitting. Currently, If you click on the Jappix logo, it redirects you to the Jappix Mini page. I'll try to make a D* help page tonight et modify the JS accordingly.

G

goob Fri 11 Jul 2014 12:32PM

Thanks. That will help you, I'm sure, to get useful feedback on your good work.

A

Anahuac Fri 18 Jul 2014 3:44AM

Hey all.. what about this: https://github.com/diaspora/diaspora/pull/5073 ? All I have to do is follow this instructions?

JR

Jason Robinson Fri 18 Jul 2014 7:22AM

@anahuac it's WIP still but sure, if you want to try that is awesome, to help the developers. But just don't expect it to be stable with d* :)

ST

Sean Tilley Fri 18 Jul 2014 4:47PM

@anahuac It might be a good idea to reach out to the author of the pull request in case you have any questions. :)

DU

[deactivated account] Fri 18 Jul 2014 5:51PM

@anahuac : Not sure the patch is still up to date since the code has changed.

H

Poll Created Mon 27 Oct 2014 2:08AM

Create an integrated XMPP client inside Diaspora and encourage thus to the podmins to run a XMPP server for their pods. Closed Mon 27 Oct 2014 9:02AM

XMPP is a good choice, known by all us... diasporabrazil.org has implemented the XMPP chat with great results. I think that if every pod runs a XMPP server, it could function in a client integrated on the Diaspora instance. We do not need Heroku or any third party's centralized server. We have many pods that can execute XMPP.

If we make a XMPP client integrated to the site, the podmins would be encouraged to run a XMPP server...
Look at the diasporabrazil.org XMPP solution at the #diasporabrazil tag.

Results

Results Option % of points Voters
Agree 71.4% 5 T KAK C JS MS
Abstain 0.0% 0  
Disagree 0.0% 0  
Block 28.6% 2 N DU
Undecided 0% 258 BK ST FS MS TS AA S CB HF BO JH DM GC JH JR F RF M EG G

7 of 265 people have voted (2%)

C

Crazypedia
Agree
Mon 27 Oct 2014 2:48AM

Due to XMPP having a federated structure already, support for encryption, and being an already fully developed protocol/service, it is an excellent choice to integrate into diaspora.

KAK

Karthikeyan A K
Agree
Mon 27 Oct 2014 6:18AM

we must have having one click shell script or something to do it all.

N

NicoAlto
Block
Mon 27 Oct 2014 7:57AM

Please close this proposal as chat is already in development. We don't need another vote, we need to finish the development.

DU

[deactivated account]
Block
Mon 27 Oct 2014 8:58AM

Please close that vote. Decision has already been made.

DU

[deactivated account] Mon 27 Oct 2014 7:32AM

Is it a joke ? diasporabrazil.org hasn't implemented anything. It has just installed the project currently and officially developped by diaspora*. There already is github repos for the development of an XMPP chat. See here for front-end and here for back-end. Chat has been added to next-major release. Just look at a few posts ago. I don't see the need for a new vote here...

RF

Rasmus Fuhse Mon 27 Oct 2014 7:46AM

That's what I also thought, Augier. We don't need another proposal but a qualified pull-request.

DU

[deactivated account] Mon 27 Oct 2014 7:48AM

@rasmusfuhse : The pull request exists also thouh it is not mergable yet ;)

H

Haiku Mon 27 Oct 2014 8:01AM

It will be used Heroku as XMPP server? It seems that there will be a centralized server with the possibility of make a new server from each node, but no as default. If it were thus, we will lose descentralization...
It's great that the project is in development, but maybe you can consider my worry.

DU

[deactivated account] Mon 27 Oct 2014 8:38AM

It will be used Heroku as XMPP server? It seems that there will be a centralized server with the possibility of make a new server from each node, but no as default. If it were thus, we will lose descentralization…
It’s great that the project is in development, but maybe you can consider my worry.

As I understood, problem of Heroku is that port 6222 (usual XMPP port) is blocked. So pods hosted on Heroku won't be able to run an XMPP server.

All other can. Chat is already used on sechat.org, @lukasmatt's own pod and diasporabrazil.org. I think there is one or two others. When it's finished, all dispora pod wil have an embedded XMPP server in the form of a gem. Up to them to enable it or not.

So no, there's no centralization to worry.

H

Haiku Mon 27 Oct 2014 9:01AM

OOh, thanks for your explanation and your contribution to this project, I was waiting for that function and wondering when it will be applied, now I know that the development is advanced and as I was thinkin, with XMPP as server and client on each node. Thank you again and sorry for make a new proposal, I wasn't understood...

DU

[deactivated account] Mon 27 Oct 2014 9:03AM

No problem.
Take time to read the last few posts next time ;)

H

Haiku Mon 27 Oct 2014 9:09AM

Yes, I'm new on loomio and don't saw that was more posts. It would be great also if you post the advances on #diaspora-hq to avoid these mistakes...

DU

[deactivated account] Mon 27 Oct 2014 9:14AM

Well, I'm not sure if it's time yet for an official post. Though chat has been added to next major, its release is not certain. This is a big feature that needs to be tested before official release. It should be a big disappointment for user if it was eventually postponed.

H

Haiku Mon 27 Oct 2014 9:20AM

ooh, there's no release date? well... at least, it's in development.

DU

[deactivated account] Mon 27 Oct 2014 9:25AM

Yes, there is. Next-major should be coming at the beginning of 2015 if I'm not wrong. But next-major also contains vital improvements to the federation. So if chat isn't finished until then, it won't be possible to postpone next-release.

F

Flaburgan Sun 16 Nov 2014 10:43PM

Good new, the chat is now merged in develop branch. But it needs to be polished, so that doesn't mean it will be launched in the next release.

Y

y.semin Mon 17 Nov 2014 6:42PM

Excellent :-)

S

Shmerl Thu 18 Dec 2014 1:43AM

A very good article which might be interesting to the participants of the discussion:

http://www.html5rocks.com/en/tutorials/webrtc/infrastructure/

DU

[deactivated account] Sun 11 Oct 2015 3:46PM

i don't know if this is the right point for this, but anyway my question:
how big would be the effort of enabling webrtc in the chat (as it is already included in jsxc?