I think WebRTC is THE solution because really powerful (look at this demonstration).

As @dennisschubert said, the technology is not mature yet, but will be available for every Firefox and Chrome users this summer. I think that 80% of our users use one of these two browsers, as we do not currently support IE and Safari+Opera have small % of users on desktop, for me, a release with WebRTC during the summer is something acceptable. But we are here for debate :)

I think that chat is one of the features which has been put on hold until the Diaspora API has been finalised and made stable, but one of our more technical members will be able to confirm whether or not that is the case.

But it may well be a good idea to discuss now what will be the best way to implement chat, and which protocol(s) will be most appropriate, now so that those decisions have already been made once it's time to start coding the chat facility.

WebRTC is an JavaScript API. It doesn't mandate any protocol which you can implement through it. For chat there is no need to build server functionality into Diaspora itself - it would be crude engineering and would be too restrictive. It's better to take ready and robust solutions (for example ejabberd), and to run the XMPP server alongside Diaspora pod (let's say on the same host). Diaspora just needs a UI to communicate through it. And if user wants to use standalone client - it will work as well, since XMPP is a standard protocol.

For media, just use XMPP/Jingle as usual. If you want to work on Jingle over WebRTC - be my guest, but I don't think there are any ready solutions, and this will require a lot of work potentially. Also WebRTC will require support from XMPP servers, since you can't simply route Jingle over WebRTC relying on clients alone.

ther was an old work of a diaspora video-chat made by vittorio cuculo: https://github.com/vcuculo/diaspora/wiki/Setup-chat-and-videochat i can contact him if he want to contibute again , it's an italian user.

while i'd love to see this i think it's something that should be focused on at a later date when other things like federation are all sorted. The back end to things like federation are likely to provide a better interface for something like this.

@ruxton Yes of course, we can always delay everything to a later point of time, but then we'll never come anywhere...?

@shmerl first of all, you're wrong about your assumptions on WebRTC. it is in fact a peer-to-peer chat API, which is why it's so interesting to this project because it's decentralized, e.g., your chat messages never touch the actual web app server. in fact, the diaspora app is just to facilitate logging in and using the WebRTC service, if that's how you're instant messaging.

I like the idea of XMPP. I feel like it may be a neat experiment to see what other kind of data we can federate. The problem with XMPP is that it requires another server to be set up and maintained. It requires people who are already having trouble getting a Rails app installed to have to install/configure ejabberd or an equivalent server. Additionally, podmins hosting on Heroku are out of luck, because you can't run processes on any port other than :80, which is the one we need for the web app. So Heroku users would have to pay for another VPS just to house their XMPP server, or find space on an existing VPS for such a task. While I agree with you here that XMPP might be a more feature-rich and "trusted" solution to our problem, there's no question that using XMPP would increase the complexity and reduce understandability for many new developers. Which is why we've been so nervous about implementing it. The last thing Diaspora needs is its podmins feeling alienated.

I personally think that if our goal is chat, we should focus on some sort of decentralized solution, which is looking every day more like WebRTC. If our focus is on changing the way we federate data and getting chat "for free" along with it, then XMPP seems to be the way to go.

I would also wager that since we aren't responsible for any IM traffic over the Diaspora server(s), that WebRTC is "cheaper" solution than XMPP, simply because we don't need a central server to facilitate communication between two people.

@Tom Scott: No, I'm not wrong about WebRTC. It's not a chat API. It's a JavaScript framework for RTC media streams. See https://en.wikipedia.org/wiki/WebRTC
It doesn't dictate the signalling protocol (which can be Jingle or SIP for example). If you use XMPP, Jingle is the natural path for signaling protocol (which should be routed through WebRTC if you want a JavaScript browser solution).

Using XMPP should be a solid must, otherwise you'll be proliferating non interoperable IM networks. No thanks - there are tons of these out there which can't talk to each other. We don't need another one, just to put it into Diaspora. Better even, if someone is so desperate to have an IM, then take a normal standalone client, connect to any of many existing XMPP servers - and that's it. No reinventing the wheel needed.

I understand the issues of running an extra server. But it only makes sense to do so, since it clearly separates Diaspora from IM service. Yes, Heroku is not an option. But Heroku limitations and idiosyncrasies should not dictate Diaspora design and limit your choices. Just don't use Heroku of you need an accompanying IM service.

@shmerl "Just don't use Heroku"? There are a lot of pods already on Heroku (disclaimer: mine included). This is an unsustainable suggestion, and it's kind of a big "fuck you" to all of the podmins who are hosting free/personal pods on Heroku. I'm not going to put others through that just because I want to use Diaspora as my IM provider.

"WebRTC (Web Real-Time Communication) is an API definition being drafted by the World Wide Web Consortium (W3C) to enable browser to browser applications for voice calling, video chat and P2P file sharing without plugins."

To me, that indicates that WebRTC does not require a server. Perhaps it isn't necessarily a protocol for chat, but I'm pretty sure that WebRTC is a peer-to-peer technology which means its data is not routed through a central server. This is a big win for us, because we are always looking for better solutions to the centralization problem. It ensures that the only 2 people who see the data are the 2 people having the conversation.

I like that a lot for personal chat communications, as long as this discussion is focused on that particular problem. What I don't think we have in this discussion is a grip on what the problem is that we're trying to solve. Are we just trying to give our users a basic chat service? Or do we want to use the XMPP protocol for something a bit beyond chat services, like perhaps as a means to re-invent the Federation Protocol?

Another disclaimer: I can easily run an ejabberd on the VPS that I already own, so that particular problem isn't something I have to solve for my own use case, but I can see how it would be an issue for others...

@paulgreindl the problem is if features are developed before federation has been resolved (and stripped into its own code layer) and before an API has been finalised, those features might suddenly become broken when the code base changes at a later date. This has already happened with a chat feature and many other Diaspora features about a year ago, which put development back a long way when many developers became disillusioned at this happening. I think it's important to make sure this doesn't happen again, and so to focus on making the underlying code base stable enough so that features then added won't get broken when other parts of the code base change.

@Tom Scott:

"Just don't use Heroku"? There are a lot
of pods already on Heroku. This is an
unsustainable suggestion

It means don't use Heroku for XMPP server. If you need one - find another hosting to dedicate to IM part of your pod. If you can't - don't use XMPP server. Heroku should not limit Diaspora design - no one forces anyone to use it and Diaspora should not mandate any limitations based on Heroku preference.

that indicates that WebRTC does not require
a server. Perhaps it isn't necessarily a protocol for chat

Exactly, it's not a protocol for chat, nether it is a signaling protocol which is needed for sensible P2P media communications (most commonly used signalling protocols are SIP and Jingle). It's up to developers do decide what to route through WebRTC. See http://www.webrtc.org/faq#TOC-Why-should-I-use-WebRTC-
As I said, it makes sense not to reinvent the wheel, and to use existing protocols. Otherwise you'll end up with another walled service which won't be interoperable with anything outside itself.

Again, this discussion has nothing to do with Diaspora federation. It's about chat (IM/media) functionality. Federating the IM part should not be restricted to federating D*. I.e. if you use XMPP for IM part, it's automatically federated to the whole XMPP network. It doesn't relate to where D* itself is federated (as a social network).

Shmerl, if that's a response to what I said, my mentioning federation is because that is one of the biggest things to be solved, and until the federation code has been put into its own layer, any future changes to the federation code (of which there will by necessity by many, possibly almost totally changing the federation code) risk breaking features such as chat, which has already happened once before. Therefore it makes sense to either fix and make stable the fundamental stuff such as federation and the API, or to put them in their own layers so that future changes to them don't affect other things, before spending too much time creating 'window-dressing' features.

I think Diaspora would struggle to survive another catasrophe in which loads of features, carefully created by developers over much time, suddenly broke because of a necessary change in the fundamental network code. We'd do well to avoid this happening again.

@Goob: Sure, fixing D* federation should be a priority I think. But I was answering to @Tom Scott, who for some reason tried to tie together D* federation and the choice of XMPP for IM component. I was saying that these are separate subjects, and should not be put together in this thread (except as you said, putting D* federation as a first priority).

@goob This is absolutely valid for federation dependent features but not for a XMPP based chat...

My opinion is that an xmpp-chat for diaspora-federation is only a matter of a user-flag that says "this user has the following xmpp-chat-account". The rest (having a web-client for the chat, having a jabberd-server on the pod or an external server/account, or using WebRTC) is not affecting the diaspora-federation in any way. So using XMPP seems quite nice to me and should be able to be done even today, when federation is not put into it's own layer yet.

Okay so this debate is more or less a choice between XMPP and WebRTC. As a Mozilla contributor, I'll contact some WebRTC guys asking if they can come here because I think we do not have enough knowledge to take a decision.

Anyway, here are the arguments for the moment:

Maturity:
XMPP is far more mature and supported than WebRTC, no doubt. WebRTC works on only few browsers for the moment, and this is a problem. Moreover, specifications can change because the techno was not released in stable for the moment.

Interoperability
XMPP is used by a lot of network (Jabber, GTalk, Facebook...). Unfortunately, Facebook is completely closed and Google started to close gtalk too (you cannot add a gtalk contact from another xmpp account for few weeks now). Anyway, there are a lot a websites / services (jabber, movim, libertree, etc) which used XMPP, far more than WebRTC and there are desktop clients for XMPP too. So for the moment, XMPP wins. In the other hand, it's possible that WebRTC becomes the replacement of XMPP in the future...

Complexity
About development, we do not know enough WebRTC to compare to XMPP implementation.
About pod administration, has said, XMPP is more complex than WebRTC. It needs another server, more sysadmin stuff, more server bandwidth / power, special configuration (ports other than 80)... and is not completely decentralized as it needs to use a server.
Remember, in a perfect Diaspora world, everybody has his own pod under his bed, in a raspberry. I see the server part becomes more and more complex with the time, and I don't like that. WebRTC totally solves this problem by processing between browsers directly.

Features
Here is the killer point in favor of WebRTC. Chatting, audio, video, file sharing, conf call... (remember, this is "the" killer feature of g+, officially...). This is AWESOME.

Moreover, we are in a moment where companies are trying to put social in them. If we add such features, Diaspora can be interesting for them. And companies can put money / developers in the projects which interested them. This is a really important point, we can maybe find a business model here (and we need one!!).

A big huge +1 for XMPP chat support.

In addition to an actual chat UI, we only need to add a field in the user settings about the users XMPP handle. If the pod provides one, the user can opt to use that one. If the user does not like it or has an existing one, they will probably use that one.

I would probably use my Facebook XMPP handle since that is where I do most of my chatting with my friends.

We have a gazillion of things to do - let's not start building a platform for chatting as we can leverage on XMPP for that and just offer a simple client in the UI AND the possibility for podmins to run a server tied to their pod. If they want, Heroku podmins don't need to do if they don't want. Or they can run one on their VPS.

Personally I don't need an additional chat handle - allowing Diaspora* users to contact me easily via my existing XMPP handles would be cool though.

I would probably use my Facebook XMPP handle since that is where I do most of my chatting with my friends.

But by doing that, you will not be able to talk to me who have a Jabber account, aren't you?

Btw, integrate in Diaspora an existing account could be a good idea, but how do you add contacts? When you start sharing with someone, it automatically send a request to his xmpp account? And how the other user answer? The asymmetric relationship of Diaspora is really different of the symmetric relationship in xmpp..

OK maybe Facebook XMPP based chat engine was a bad example. But anyway, something existent which is widely used, like @shmerl has been saying

@Flaburgan:

Okay so this debate is more or less a choice between
XMPP and WebRTC.

I think you are missing a point. XMPP and WebRTC are incomparable - they are different level frameworks. XMPP is a complete IM protocol, with signalling extensions like Jingle, which use P2P RTC for media chat (video/audio).

WebRTC is not a protocol, it's a low level JavaScript API which allows using RTC from the browser. I.e. it doesn't dictate any protocol whatsoever.

Therefore it's not a question of XMPP vs WebRTC. It's a question of what protocol do you want to use for chat? And if you use WebRTC for conferencing part of that chat (which makes sense for browser based client), what protocol do you want to implement through it? Remember again,WebRTC is not a protocol. See it as a tool, through which you implement your media protocol in JavaScript.

The way I see it - full blown and feature rich chat should look like this:

1. IM component should use XMPP protocol (for example using strophe.js preferably with WebSockets, instead of BOSH).
2. Conferencing component (video/audio chat) should use XMPP/Jingle signalling protocol, utilizing WebRTC framework to route it from the browser.

I hope this clarifies things a bit.

Anyway, my advice to anyone who proposes to use WebRTC in any way - read first about what it is, and what it isn't.

@Jason Robinson:

I would probably use my Facebook XMPP
handle since that is where I do most of my chatting
with my friends.

I agree that users should be able to set their XMPP handle, which can be from another server, not necessarily from the server shipped with the pod. However it has to be a federated server naturally. Facebook XMPP server is not federated, so we can forget about it right away. The fact that it's XMPP becomes irrelevant, since it's cut off from the XMPP network.

Google Talk is federated. Their invite blocking problems were temporary and they already resolved that issue.

I'm running a XMPP server and a Diaspora pod side by side at wk3.org, so the users already have matching handles.

Since nobody came up with SIP yet, I'd like to suggest that. There's already a HTML5 client: sipML5 And we totally need to consider CU-RTC-WEB too.

SIP is fine for media, except that it doesn't integrate with XMPP, instead it integrates with its own IM protocol - SIMPLE. However Jingle is a natural extension of XMPP, and if you are already using XMPP for IM/chat, it makes sense to use Jingle rather than SIP.

Like Shmerl said, WebRTC is just a channel that can be used for any kind of content. Yet it will be easiest to implement video-chat and phone-calls since it has the best developed libraries.
My question would be (and I'm not sure what the answer is): is it possible to fully mimic an XMPP with webRTC? There are some asynchronous components in XMPP that would need to be done by the Diaspora-server like "does the user exists", "give me metadata of the user" and "send a friendship request", probably more. I am quite sure that this can't be done entirely just by using WebRTC.

But it would be definitely a good architecture, if the user can opt if he/she wants to use a) an existing XMPP-account, b) a provided XMPP-account by the pod (like Christophe's solution) c) a WebRTC-client for XMPP or d) no XMPP account at all.

If we want to focus not only on chatting, but also on video-calls and this stuff, we should consider this probably now, because this can be hardly be done by XMPP only. I have no idea what would be the way for that.

I see this discussion finally moving in a good direction :)

@Rasmus Fuhse: For text messages you don't need WebRTC. It's a clear overkill. JavaScript offers you WebSockets already. WebRTC is targeted for media streams (i.e. audio/video), that's why it's dealing with codecs and the like. WebSocktes is a generic API for duplex communication through JavaScript. It fits perfectly for text part of XMPP. Some support for that is required on the backed (before WebSockets, the only method from the browser was BOSH). Now servers like ejabberd are catching up with WebSockets support. For media, WebRTC is an obvious API, but as I said - the protocol still needs to be defined, and it makes sense to use Jingle there:

#1. Text XMPP can work like this:
JS client (browser 1) <-> XMPP over WebSockets <-> XMPP Server <-> XMPP over WebSockets <-> JS client (browser 2)
#2. Media XMPP (Jingle) can work like this:
JS client (browser 1) <-> Jingle over WebRTC <-> JS client (browser 2).

Except that in #2 you still need some support of such routing from the servers, since there are security limitations. See some discussion in the comments here:

https://hacks.mozilla.org/2012/09/full-webrtc-support-is-soon-coming-to-a-web-browser-near-you/

The benefit of using XMPP and Jingle is that instead of JS client you can put any client (Pidgin, Jitsi etc.), and it would still work, since it will talk a standard protocol.

@shmerl If we are speaking of adding XMPP as an IM service only, then we may have a reasoning behind this. We can use XMPP for a lot more than just IM, if we were so inclined. I believe both Facebook and Twitter are/were using XMPP for realtime communication between their client side and server side code.

If we do this, I'd like to give the user the freedom to run XMPP or not. If they don't want it, they can disable its use. If they do want it, it is their responsibility to provide a server and XMPP protocol support on the backend. Diaspora is essentially just providing a nice web-based client to use with an XMPP server.

For a first version, I feel like it'd be easier to just focus on text chat, and then move on to voice/video implementation later. Are there free software implementations of this on the client side I can take a look at? Also, what is everyone's consensus on using a Flash component to handle video chat for regression into older browser clients?

@Tom Scott: Yes, we can use XMPP for Diaspora's own protocol, but as I said, this is out of scope for this particular discussion.

Back to IM: XMPP surely can be optional. Podmin can simply not associate the pod with XMPP server (and not install that server). There is no need to make it mandatory.

Regarding clients - Text/IM libraries are already around. Example:
http://strophe.im/strophejs/
https://gist.github.com/processone/739147

Audio/video conferencing and Jignle support through JavaScript is still catching up and you'll have to do the research. But that can be put to phase 2, with text IM being the first phase.

Who is the person requested to implement this feature?

@christophe - chat has been one of the most wanted features since the beginning, requested by many people

@tomscott

Are there free software implementations of this on the client side I can take a look at?

Look at Jappix Mini ;)

what is everyone's consensus on using a Flash component to handle video chat for regression into older browser clients?

I'll block anything using Flash. This technology does not exist anymore :p

@flaburgan uhh, i think you misunderstood my question. but it was a stupid question anyway, because i'm pretty sure we can get websockets to work all the way down to IE 6 without using flash.

@shmerl i do know about Strophe. should be a decent choice for communicating with the XMPP server. let's focus on text chat right now. would Jappix Mini or the Facebook IM design work for our purposes?

and by "Jappix Mini" i mean the way it looks, not the product itself. we can't really use Jappix as a product because we don't control its code.

How would you link Diaspora accounts with XMPP accounts?

Do you intend to allow users to provide an existing XMPP account or should they register a new account (on the connected XMPP server) through Diaspora? (see XEP 0077)

@rekado when i was playing around with ejabberd, i found the easiest solution would be to simply create an XMPP user for each diaspora user on the pod. and when new users are added/removed, we create/destroy their corresponding jabber user. seems pretty easy since both of them are using a relational DB.

@Tom: I would advise against modifying the DB directly as you'd couple Diaspora too tightly with a particular XMPP server implementation (and maybe even version). XEP 0077 is the standard way to register accounts through XMPP. (My apologies if this is what you meant.)

Hi to all!
This is my first time I use loomio so please have patience!

I don't know how much voice and votation I have, but I give my personal opinion about this:

I met lot of people that wants to have a chat feature, not a voice call neither video calls. Also I met people that just said "ah! that doesn't have a chat: it's boring!" so I cannot convince him to use D*. In other words: people got acustom to chatting instead of talking.

One problem about voice and video is not the implementation, is the Internet "speed" and bandwith. If you're going to add voice or video, please be sure to make it clearly optional.
If you ask my opinion about this, we should concentrate on chat. Video and calls are too much and has its own consequences(bandwith, services, privacy, etc.). At least, we should have a chat!

About XMPP, it do support creating counts through its protocol(I remember Pidgin asking if I want to create the account or use an already created). Then, the problem is if ejabberd or jabber.org will may be overpopulated and if they can stand so much people registered.
This makes clear why should be a separate service and not hosted in Diaspora servers, maybe some pods cannot stand such load. Diaspora can give an easy and beautiful
web interface, but the rest is up to jabber.org(and we support Jabber!).

I haven't take a look into WebRTC, but for now I vote in favor of XMPP: is mature enough for give more robustness to D*.
Cheers!

@christiangimenez - everybody has a voice on Diaspora* :) Welcome!

@rekado at this point, i think just supporting a single XMPP server would be sufficient for our needs, but regardless if we can use a standard to make sure that our code is "future-proofed" for any other standards-compliant XMPP server that gets invented in the future, that seems like the "right way" to go.

@rekado to answer your question, i think XEP-0077 (http://xmpp.org/extensions/xep-0077.html) is a fine spec to go by when registering accounts on the XMPP server. there is an XMPP lib for Ruby, so we can send messages to the configured server like when a user is registered through the web interface.

Actually, there's more than one XMPP lib for Ruby. I recommend using blather (rather than XMPP4R), because of its cleaner design thanks to running inside an EM loop.

so, this is the direction I see this discussion going:

D* associates a Jabber ID with a D* user. That could be a user-specified, existing JID - or - D* offers to create a Jabber account on a user-specified server via XMPP protocol (could have a nice auto-complete for a few known servers).
(the password would need to be stored in the DB unhashed, but we could at least encrypt it with the users private key)

The UI offers a 'chat' feature, that's more or less simply a Jabber client in the browser, connected to whichever account is specified in the D* users profile.
The exact implementation details are still to be decided...

@florianstaudacher I just signed up because I wanted to write exactly what you already said ;-)

Just adding this: When user A starts sharing with user B the JID of A could be send to B. When B also starts sharing with A (so both know each others JID) we could add the JIDs to the jabber contactlist

It appears that many existing XMPP users will have to either get new accounts or install BOSH on their server to use such a feature.

@steffenvanbergerem do you mean that user A adds B or just display it?
If adding, D* should ask in which group to add B.
This is useful when we're talking that D* server forward messages to Jabber servers.

@sethmartin what @florianstaudacher mean is that D* will only associate an XMPP account to the user, and then will only act as a client(maybe using Javascript or other).
If you have an existing account you associate it with your D* user.

I suppose that there's no need to alter the server functionality, the problem reduces to find a good free jabber client to embed into the webpage. Or, if I forgot something about securities issues, the server just act as a passarel, fowarding messages to the Jabber server.
Am I right? :-S

I found lots of free(as in freedom) Jabber clients and libraries written in JavaScript: JSJaC, Candy. Take a look at the Jappix Mini feature looks interesting...

Hope this helps a bit more...

@christiangimenez all of these require BOSH because that's the way to write XMPP clients for the browser....

You're correct, it use BOSH. I learn something new! :-P

If you were fine with the idea that users had to create a new XMPP account through Diaspora, then you would not need to use BOSH and the JS side of things could be rather simple (= no need for a full XMPP client in JS). Instead of distributing a full client to the users, you could push messages through a websocket in either direction; the websocket server process then takes over the role of an XMPP client.

The benefit here is: no need for BOSH, smaller JS footprint, easier to customise the chat interface, you can use Ruby + blather to interface with the XMPP server in an event loop rather than using the chopped up XMPP-via-HTTP thing that is BOSH.

yeah sure, that way we could achieve much tighter integration with our application, but it would also increase the work required to do this feature.

Also, I'm no expert on Rails but the googling I have done lately indicated that Rails is not particularly good at doing anything else apart from HTTP request-response cycles. Websockets and all the other nice tricks require a separate event loop that's outside the Rails app, meaning additional ports would be needed to be opened for D*. (Please correct me, if I am wrong.)
So what I'm saying: a standalone chat server in ruby appears to be no problem, but integrating one in D* - even if it's just to relay XMPP messages to the client browsers - is a completely different story.

Websockets and all the other nice tricks require a separate event loop that's outside the Rails app, meaning additional ports would be needed to be opened for D*

To support websockets you only need to run a websocket server. This is a simple, separate process (e.g. written in Ruby) that dispatches on messages that come in via websocket. This process can (and should) operate independently from the Rails application. Thanks to library support for websockets, it's very simple to make this work. The separate process could be spawned during initialisation of the Rails app, so it doesn't even need to be started separately.

One additional port is needed to be open, but I don't consider this a problem --- it's certainly much less of a problem than installing and configuring an XMPP server.

BOSH is not the only way, and not a preferable one. Better one is Websockets.

As for Heroku - that will require to come up with XMPP, I'm not sure that Heroku is at all geared towards that. It seems to be targeted only to web applications, not to other type of servers. So pushing XMPP server in Heroku would seem like using a wrong tool for it. While it's good if that could be possible, but any kind of approach with it would be not straightforward. Much cleaner solution for those who rely on Heroku is simply to find another host where to run their XMPP server.

Just to clarify what agreeing to this proposal means: "The WebSockets protocol is not yet supported on the Cedar stack." https://devcenter.heroku.com/articles/using-socket-io-with-node-js-on-heroku

It's also impossible to open arbitrary ports, so running an in process XMPP server is also impossible.

Thirdly I wouldn't consider requiring a third party XMPP server as "working on Heroku".

long-polling/comet! ;)

I don't think our issue is S2C. Our problem is S2S.

Jonne Haß: Then clarify please what kind of options for XMPP server are possible on Heroku? From what you said, it sounds like they are non existent. So what is the poll about then?

Although it's not ideal to trust yet another party with your data, third party BOSH can also be used for the few XMPP users that don't have BOSH. IMO the ideal solution here would be a simple Heroku compatible Jappix mini type plugin like Friendica uses.

That's what I want to clarify. I've not come over a decent solution that allows using XMPP on Heroku without addtional pain/costs for either the users or the maintainer. From my current knowledge it's impossible. But then a large percent of our userbase is on Heroku, one can only guess the numbers, but I'd be surprised if it's below 30%. And we can't say these 30% are on a well maintained and committed pod currently, sadly. While I didn't read every word in this discussion I saw a clear tendency to use XMPP, possibly over Websocket or BOSH. Again this is not possible on Heroku.

So we need to decide if we want to further seek for a solution that works on Heroku or allow one that doesn't work well there. Or we won't move forward in this discussion. That's what this poll is about, support chat on Heroku or not.

(That "possibly using XMPP is in the discussions title is probably a bit confusing and should probably be changed. Nonetheless it's an option and not decided yet.)

If supporting Heroku means not using XMPP for the chat - then I'm voting No. My Yes vote was only for the effort to find how to use XMPP on Heroku.

I'd propose to create a poll for using XMPP as a solid base for chat protocol then. And see what people say.

I agree with what Seth Martin said:

I'm not interested in any other solution than
XMPP because I want to take my chats with me
when leaving the browser. I hardly consider it
screwing Heroku users by requiring they use a third
party XMPP server if they want to chat.

@shmerl Please edit your vote then.

@rekado blather looks interesting...honestly i had no idea anything other than xmpp4r existed as a library. wasn't too excited about using something that hasn't been updated in 3 years. ;-)

I imagine this whole XMPP feature would look something like the setup for an email server. In which you tell it what server to connect to, and what username/password to use. You can also perhaps enable or disable the "sync accounts" feature which would basically create an account on the XMPP server according to XEP-0077. This would enable users to use their existing XMPP server on Diaspora IM.

I think what most people here really can agree on is that they all want a web-based chat UI inside Diaspora. For some, this may mean a secure XMPP server for sending messages that they control. For others, it might mean just hooking up their jabber.org account and simply using the UI. We should make each option available.

Loomio isn't taking my vote. Though to me the options are broken. XMPP will be split in no and abstain votes, while non-xmpp gets all the yes votes. I think we should count no/abstain against yes, the options are not well thought out.

Anyway, seriously, are we going to plan the feature set of Diaspora* (free, open source, privacy aware etc) on the whims of one proprietary company? If there is a better option than XMPP then let's hear it - but this HDD (Heroku Driven Design) talk is just mad.

Of course we should allow pods to run on Heroku as easily as possible but it should not control core functionality design.

No problem in counting the majority as absolute one, i.e. not reached if not > 50% of all votes are yes. In fact that was my original thinking.

I don't want to fight for Heroku. I want to know whether to support it or not. I see the discussion stuck otherwise.

Free at herokus means a starter database with upto 10k rows? Because of the current federation system and duplicate posts you reach 10k very soon, hence you need to pay for running diaspora on heroku. Who is running diaspora on heroku for free?

I know joindiaspora.com runs on heroku but why not using an external xmpp server on another machine?

We should find a solution for chat, not a solution for heroku!

Even if heroku is not allowing XMPP, our decision is not: XMPP or not XMPP. We could also implement a fully working XMPP-chat with a per pod option to disable XMPP-chat for the pod/it's users. Users of the pod wouldn't be able to chat. But that is due to limitations of the crappy Heroku settings. Users of better servers will be able to chat with each other.

We shouldn't let Heroku dictate us how we are using our network. But on the other side we should create our software in a way that it runs even on low-profile machines. We don't need to implement an evil XMPP-hack for Heroku, but we need the option to run a non-XMPP-diaspora-account.

Then, I don't get very well the idea of the Yes/Abstain/No options. What does these means then?

If yes means implement it on Heroku, then I have to disagree or abstain... Heroku is no important mather as long as XMPP works. Unless we agree that this is just a start until we decide another better/free/descentralized alternative.

We have to consider that we shouldn't limite the people runing pods to use Heroku. I mean, if I want to start up a pod, I can do it in my own work/home without any Heroku's connections.

I can see that we all agree to use XMPP. Is a semi-descentralized protocol and that is atractive! That is cool! :-)

@jasonrobinson I don't think it's so much about bending to the "whims" of Heroku, so much as it is the reality that many Diaspora pods are already hosted there. Crippling a featureset for users on a specific host is just as bad as bending to the whims of that host in regard to development.

I think going forward, we need to consider alternatives to Heroku, if we really want to encourage podmins to move over to be able to benefit from chat. I have heard that OpenShift has some promise, and @jonnehass did some work a while ago making diaspora work with OpenShift.

Basically, if we want to develop XMPP-powered chat and not worry about Heroku, we should come up with a good alternative to it that can better suit podmin needs in regards to these kinds of features we want to do. Otherwise, we'll need to find something else that works for everybody, regardless of what host they're on.

@seantilleycommunit implementing a chat solution that is optional but doesn't work on Heroku will not cripple the existing user base. They just will not get the new feature.

I really do believe we should not base any design on a single hosting solution. Be it Heroku, Openshift or any other. Let's find the best solution out there, make sure it works on as many platforms as possible and then decide on that. Be it XMPP or something else.

@diaspeu you're misunderstanding how that works. the database doesn't lock, it merely begins deleting old records. perfectly fine for a social network that is more about what's happening right now than what happened 5 years ago. for example, what i wrote on Facebook 5 years ago is not only meaningless but terribly uninteresting to me today.

As I've said before, and I will say again, if we do this I volunteer to either code it or head up the team that's tasked to make this all work.

@tomscott I got it, but at diasp.eu it takes 6 days to get 10000 posts (approx. 6000 from joindiaspora.com), means my last post, 7 days ago, will be deleted :-) Take a look at https://diasp.eu/stats and see how the federation system fill up my db.

@diaspeu haha makes sense, you have a public pod. mine is private and just for me. having a free heroku instance run your pod makes sense for people who don't use Diaspora that often and just want to control their own data. in my case, i've always just liked playing around with the code, so it's nice to be able to use my own changes.

@tomscott Your pod at https://s.psychedeli.ca/posts/1 running an old version and I'm sure that your workers are dead because you don't receive any posts from other pods.

If you run diaspora as intended with workers to receive posts from other pods you db will fill up quickly and thats the reason why this will not work on heroku for free :-)

(juste about the "WebRTC is not available yet", Mozilla put it on by default in the last beta, so it'll be in stable in 6 weeks.)

I really do believe we should not base any design on a single hosting solution.

Everyone will agree that. But I have another point of view: I'd like to see Diaspora light. I'd like to run it on my Raspberry Pi, with only a "apt-get install diaspora" that the package team will provide. And for that, I have to be the lightest solution possible for the chat feature: my pod should do the minimum. For me, integrate an XMPP server in Diaspora* is not a good solution because Diaspora will become too heavy.

Keep things simple: a setting for the podmin to indicate an XMPP server which will be used by Diaspora*. This server can be installed in the same server, in another one, or maintained by someone else like jabber.org, but will not be integrated inside Diaspora, which means you are not obliged to install it in the same host.

Does this solution fit with Heroku? I don't know.

The other solution, more powerful, lighter, and which works on Heroku, is to use WebRTC. But it's less mature, cannot be used outside browser for the moment, and is not linked to our already existing XMPP account (can't talk to gtalk, etc.)

@Flaburgan: I think we should stick to interoperable protocols as a priority. So creating custom made protocol through WebRTC is a bad idea by default. "Lightness" can be achieved by not requiring XMPP server to run on the same host as Diaspora.

I'm sorry about Raspberry Pi, but it's completely not suitable to run an XMPP server, let alone any other serious server with multiple concurrent connections, but I doubt anyone would want that anyway, so in such scenarios D* pods will use remote XMPP servers.

@shmerl i completely agree, this was my point: I don't want to integrate an XMPP server to Diaspora*, I want to make it possible to run in another host.

I think a Rails Diaspora* will always be not available on a Raspberry Pi, due to memory use ;)

@Flaburgan: it's probably not wise to mix this proposal with the idea of running Diaspora on ARM. There are quite a few other problems you'd have to solve first (e.g. database) before chat would become a problem. If you'd run the database server on a remote machine for performance (leaving the Pi as a control centre only), then the same thing should be done with an XMPP server. Just put it on a separate machine in the same network as the Pi.

@rekado running Diaspora on Pi for a single user pod is entirely possible. It's slow but possible and yes, I kept the DB on the Pi too. But we're really getting offtopic here.

Just in case it's not clear for everyone: if somebody wants to start implement a instant messaging functionality based on XMPP, I'd be able to mentor the XMPP part of the whole process. Just contact me..

What are the next steps?

I guess, it's quite clear how the chat should be implemented now. Next step should be to find someone who implements it. It's your proposal actually.

Is it too whacked to think that a hack like the auto-update via greasemonkey could do it alternatively? I'm thinking of a simple checkbox at the bottom of the conversation view, so that you can let it check for new messages every tenth seconds or so if you're having a hot debate with someone?

Well, it's not a real hack. It would absolutely work. but still we had to implement a federation and stuff. And when we want to do so, we should most likely implement XMPP instead, because it's an open standard.

I prefer XMPP, because we could maybe connect it to WebRTC, so we have a really big feature coming.

http://hancke.name/jabber/webrtc-xmpp-summit-13ph.pdf

@diaspeu the workers aren't running because the SSL cert is fucked up right now. I never really had too many friends on Diaspora in the first place so my profile never got much activity even when it was on joindiaspora.com…which is why I thought a free Heroku instance would be sufficient for a presence. I really mostly use it to test stuff that I'm working on personally.

I'll try and get the SSL cert working again to see if running it on a free Heroku instance is as bad of an idea as you say it is. =D

@dennisschubert I would love to work on this with you. are you around much on IRC?

@tomscott cool! sure thing, my nick is 'denschub'. please note I'm living in CEST (but I'm usually working all night long, so that should be no big deal). alternatively you can reach me on XMPP, [email protected]

(eh, denschub on freenode, of course.)

+++ Thanks to all of your comments and support in this story. Now we come to the important point, the implementation. Let us work together and make diaspora a better place! +++

@Dennis, @tomscott: what are the components you'd implement? Are you going for a full XMPP client in JavaScript that registers with the local XMPP server (e.g. via BOSH) or a minimal connection between browser and dedicated XMPP backend (e.g. websockets that push messages through to an XMPP server component which is connected to an XMPP server---local or remote---through XEP-0114)?

Although I have not talked with @tomscott yet, I think we're going to implement a full XMPP client based on Strophe.js. That will take a bit more work than just passing the stanzas through some other channel (as we have to write a little auth provider to authenticate with your Diaspora user credentials), but that's the cleaner solution. IMO.

We should also consider just storing a JID and a password and running an open BOSH proxy like https://github.com/dhruvbird/node-xmpp-bosh as a fallback setup.

Full XMPP client can use Websockets as well which is preferable over BOSH. But some XMPP servers are lagging behind with supporting Websockets (for example open source ejabberd didn't support them the last time I checked, but commercial one did). I'm not sure if that changed recently.

@shmerl:

But some XMPP servers are lagging behind with supporting Websockets

I don't think it's something they even should support. It's not like that is defined in an XEP somewhere. (There's a third-party draft for XMPP over websocket, but it's not anywhere near a standard, so one can't expect XMPP servers to support websockets natively.)

To shuffle messages from the frontend to some Ruby backend via websockets is trivial. The Ruby backend could easily manage the necessary state (e.g. Diaspora authentication and other plumbing) which would otherwise be rather awkward in JS (the Ruby process could talk to the db directly, unlike some client-side JS which must take a detour through an HTTP API).

(I've been using blather over the past weeks to turn Libertree into an XMPP server component.)

It's good if they support Websockets the same way it's good when they support BOSH. I.e. it's not really necessary for pure XMPP over TCP/IP, but for web browser solution it is necessary, since they have security limitations and can't open free socket communications the way standalone clients can. Therefore some additional support from the XMPP servers is needed (whether it's BOSH or Websocket) to enable communication with browser based clients.

I meant direct communication with the XMPP server over Websockets, rather than with Ruby backend. It's more interesting, and there are efforts in that direction.

For the reference, the draft you mentioned is probably this one: https://datatracker.ietf.org/doc/draft-moffitt-xmpp-over-websocket/

Another thing: I think everyone here is of the opinion that real decentrability is only possible if one can simply switch to another server and take the data and contacts to the new server. This feature has jet to be implemented into Diaspora, but it's possible. But what about XMPP? I experienced that switching servers is not so easy when using XMPP.

@Trolli: XMPP is just a transport protocol. If you were to implement the whole thing as a server component, things like data migration (as well as everything else) would have to be handled by the server component. If you tie the implementation to a single XMPP server instance, however, you are subject to its implementation defects.

@Shmerl: yes, that's the draft. Note that this is not a draft published on the XMPP site and also note its status.

If you were to develop a solution that expected an XMPP server to support websockets, you'd make the feature incompatible with most XMPP server implementations out there.

@dennisschubert : There is also ongoing work for Jingle support in Strophe.js https://github.com/ESTOS/strophe.jingle
evertheless, it still seems to need Bosh

@trollischmittlauch strophe.js will always need BOSH. You cannot create a TCP stream in your browser (yet.) But we'll set up a little BOSH proxy shipped with Diaspora for all those non-BOSH servers...

@tomscott what's the status? ;)

@trollischmittlauch: Great, so Jingle over WebRTC is coming at last.

@dennisschubert:

strophe.js will always need BOSH.
Not really - as discussed below, it could use WebSockets which is even better, but it's not yet standardized for XMPP so BOSH is still preferred.

So @dennisschubert and I have decided to build auth modules for prosody and ejabberd in order to have the XMPP server auth with Diaspora and not have to maintain 2 synced databases of users. Instead, the XMPP server will query the Diaspora pod for whether a given username/password exists, and we keep all user data in one DB.

In the coming weeks, Dennis and I will be discussing how to best implement the Diaspora-side API that deals with XMPP. Since we're writing our own auth module, we are going to start with just POSTing to the Devise user_sessions endpoint, and evaluating the response.

On the client side, I'll be implementing an XMPP client from within Diaspora using strophe.js and BOSH. I know BOSH, that's why I'm using it. We can also give WebSockets a shot but I'm leaning towards the "tried-and-true" for this particular feature. The first iteration will probably not have configurable usernames, but in the future I want to add that so you can connect the web client with any XMPP server, if you just want to use diaspora's chat client. I'm not sure how that will work on pods that don't have an XMPP server, since we probably want to remove any UX related to IM on pods that do not support such a service.

Thoughts and comments would be much appreciated! <3

Perhaps a temporary workaround would be to include a configuration flag such as IS_HEROKU, that way if a pod couldn't support XMPP, the interface and related links and buttons for chat would simply not appear at all.

Also, out of curiosity, what is the current state of WebSockets? I seem to remember Diaspora supported it ages ago, but it had been shut off for one reason or another.

@seantilleycommunit definitely. since XMPP chat is an optional feature, the interface won't be loaded unless you configure an XMPP server. when the feature is in master and pods update, nothing will happen. each pod who wants to support this will have to configure their own XMPP server.

additionally, at this time we are only supporting ejabberd and prosody, since @dennisschubert knows both Erlang and Lua. if anyone wants to contribute an auth module for OpenFire that would be super-awesome. I guess we'll host these projects on the main diaspora organization in GitHub, so people can easily find the plugin they need.

I guess I'm repeating myself, but I'd like to warn (again) against developing against particular XMPP server implementations. It's work that you don't really need to do to achieve the goal, you have to track more upstream projects, and it will only work for selected versions. It also means that one cannot connect one's Diaspora installation to an existing XMPP server installation without installing your third-party auth module.

Of all possible solutions this seems to be the ugliest.

(Wow, I just used /again*/ for three times in the same sentence.)

@tomscott "The first iteration will probably not have configurable usernames, but in the future I want to add that so you can connect the web client with any XMPP server, if you just want to use diaspora’s chat client."
I beg you, please don't forget that it should be a high priority to accomplish this ability. I don't think that there are many people that want to have a second XMPP account just for Diaspora contacts. I know... one step at a time. I commend you and @dennisschubert for taking on this project and imo it appears you are making the right decisions.

With that said, I'm glad that I just revisited this discussion because I'm prepared to install an XMPP server on the same machine that hosts D*. You just helped me decide on Prosody. Something that's not yet clear to me is whether I will be able to use node-xmpp-bosh instead of Prosody's local BOSH. I wish to have the ability to provide an open BOSH for those without. Will this be possible?

@tomscott also, if I were to set up my XMPP server now add an account for myself with the same username/password as my Diaspora account, would I be able to use this account with the first iteration?

@rekado this actually came up in our discussion. i would be in favor of doing something like this: http://xmpp.org/extensions/xep-0077.html but, it was brought up by @jonnehass and @dennisschubert that keeping two DBs in sync (the XMPP server's user database and Diaspora's database) might be more difficult. What do you think?

@sethmartin from what I said in that post, no you wouldn't be able to at this time. but, we're still in the planning process. if it turns out that keeping 2 separate DBs of users is easier than doing the auth module thing, this may change.

Woah! I've just read through the entire discussion in one go. And it's nice to see that after 2 years this is still a most wanted feature for D*.
I read that you've decided to adopt XMPP, and since I've realized a working implementation of the chat/videochat feature for D* (2 years ago) I may give you some advice and helps developing it.

Server side

As @tomscott said, keeping two DBs in sync might be difficult as well as expensive ($$$).
But the XMPP server needs to access to D* database in at least 2 occurences: authentication and friends list.
So here are the solutions I've used, and I hope they may help you.

Authentication

Ejabberd offers as auth_method an external option which uses an extauth_program to check if credentials are correct.
The extauth_program should follow these specifications.
You can find my implementation here and other info here
This solution allows to use the xmpp account also with external clients.

Aspects/rostergroups

XMPP provides "rostergroups" that correspond to D* "aspects", and "rosterusers" that correspond to your friendlist.
To keep these connections my solution was to create two DB views to get rostergroups and rosterusers, that, by definition, they are updated when an edit is done on the tables.
But I'm not sure if this is the best solution (at least it was for me 2 years ago).
You can find it here, in particular rows 161-162.

Other

Other minor configurations are required to ejabberd.cfg file. Here's an example which is the same of the original configuration file but edited in these lines

Client side

Reading this discussion, I think you have already carefully considered all the possibilities. In my implementation I've used BOSH too, because the websocket module for ejabberd was still unstable (I don't know the current status).
The client side was, of course, made by two JS files, one which implements the core functionalities (that you can find here ) and it uses Strophe.js library to connect to the XMPP server. And the other file implements the UI here , and it's based on mini.jappix.

Problems

1. Consider that people may want to use their existing xmpp account
2. Consider different types of XMPP server
3. Consider that you may need to open 3 different ports (c2s for non-web clients, s2s, bosh).

That's all folks!
I really hope this might be useful for you, and can avoid you to reinvent the wheel.
For any need you can ping me here or on D*.
Oh, and thanks to @diaspeu for bring me here :)

I forgot some (hopefully) useful links:

https://github.com/vcuculo/diaspora
https://github.com/vcuculo/diaspora/wiki/Setup-chat-and-videochat
https://github.com/vcuculo/diaspora/wiki/XMPP-chat-feature

Prepare yourself, the following post is long :D

Whaouh, that's great. Thank you Vittorio.

Awesome @vittoriocuculo, so glad to have you participating in this discussion as I believe you were the one who made the initial PR with a basic implementation of XMPP on Diaspora. You fuckin rock!!

I think we could all look at Vittorio's implementation of this as an example for how I'd like features to be introduced into Diaspora. Someone forks the projects and takes the initiative to figure out the codebase and make a major change without deliberating for a long time on how it should be implemented. Even though his patch wasn't initially accepted, we are clearly learning from his experience and I'm intending on reusing as much of his code in the new PR as possible.

So, double thanks to you Vittorio! I'll do some more prototyping this week and check out your fork to get inspired.

@Tom Scott: you don't have to maintain two databases when you cheat by implementing an XMPP server component. A server component can connect to any XMPP server with its own domain. What's great about server components is that they can author messages for any user on their domain. This means that the server component could directly use the Diaspora database and map this to XMPP concepts. The XMPP server is only used for transmission and reception of stanzas; any stanza addressed to the server component's domain is not processed at all and directly passed to the server component.

In my opinion this is a much cleaner implementation (which is why Libertree and Buddycloud do it this way). You'd have to implement a mapping between aspects and rosters and wire up the stanzas you want to support. This is considerably less effort than developing parts against existing XMPP server implementations and it's much more robust as it's completely independent from any XMPP server implementation.

A server component is easily written with blather; it runs in its own EM event loop and could be spawned during application initialisation. It's self-contained and can be implemented in 100% Ruby.

I' enjoying your discussion...

Just letting you know the estos GmbH just released a strophe.js plugin to support jingle (and some fun with WebRTC if available): https://github.com/ESTOS/strophe.jingle

rekado: How easy would it be to add Jingle support to such component? It might be a big headache once you decide to do it, in comparison with working with major servers like ejabberd and others.

@rekado that sounds like the best of both worlds. is it too good to be true? :)

alright so can you write that server component? or will we need to find another volunteer? i would do it if someone takes the client-side work off my hands, because if we both work on each part at the same time and communicate properly, we'll have this done pretty quickly.

@tomscott I'm afraid I won't have enough time to work on this :-/

It is fairly simple but my next months are pretty much (over)booked and I wouldn't like to delay this feature.

XMPP server components are simple, which is their biggest advantage and their biggest disadvantage. They allow you to use XMPP servers to handle communication, but they don't actually do anything you haven't implemented. Handling chat messages and supporting presence stanzas is almost trivial. If you want to support extensions to the protocol, however, this can be more difficult. (But it's pretty difficult/messy to link an independent extension implementation in some XMPP server to Diaspora anyway. I still hold that the bare XMPP server component is a cleaner platform to work with.)

@Shmerl: you are right, Jingle is not supported out of the box. Looking around there doesn't seem to be a dedicated Ruby library for jingle. The XMPP part of Jingle is pretty easy to implement; after all, these are just normal stanzas with payloads to negotiate a connection (data transmission itself done outside of XMPP). Actually establishing certain connections and supporting various protocols is more difficult, but I'm sure there are Ruby libraries for that. The only missing piece would be the negotiation according to XEP 0166.

@rekado looking at the docs for blather, it seems like a pretty simple DSL. I could probably hack it, but it may take a little longer to get the client side portion out unless I can get some help with that stuff. I'm also not interested in extending the XMPP protocol…supporting basic messaging and authentication between the app and an XMPP server is just fine. so I think the server component written in Blather would be perfect.

i'm thinking about making this a separate Rails::Engine. this way, people can include the gem into their Diaspora installation, like a plug-in, and it would provide the XMPP server component as well as the front-end chat features. this allows people who can't or don't want to run their own XMPP server to simply disregard all of the code for this feature. a mountable engine works seamlessly with a Rails app, as if it were part of the app itself, so I think it'd be a perfect way of encapsulating this feature.

rekado: I think it's better to avoid implementing any server components, since it's reinventing the wheel and will eventually simple fall behind and become a big mess. If possible, Diaspora should handle only client side, where server can be anything external.

The issue is, what and how do you want to define as a list of users and groups in the context of Diaspora in such case. That's as already mentioned below is something to figure out. It's better to solve this issue once, rather than trying to catch up with XMPP server development, any time new stuff is added to that. Just estimate the effort.

@Shmerl: "reinventing the wheel" <--- no, everything is wrapped up nicely in a library.

If Diaspora only handled the client-side there would be no integration of the chat feature. There would be no advantage of using the built-in chat client over an external chat client.

A server component is not the same as developing an XMPP server. If you write against specific server implementations to gain some degree of integration, you'd still have to figure out how to map the server's features to Diaspora features. The difference is that in the case of writing against plugins you don't have library support and you can't write in a generic fashion. You'd have to duplicate your efforts for every server implementation that you want to actively support.

XMPP is no magic. You can't just write once and reap all benefits of all XMPP extensions that happen to be available on a given server. Not if you want to go beyond embedding a generic client in the browser window, which is the most unsophisticated and least helpful approach to adding XMPP chat support.

FYI: May be this will be useful for something: http://candy-chat.github.io/candy

With prism, many people want to leave gmail. But they use the chat and miss it. Having a chat in Diaspora is one argument more for adoption.

What's the progress on this guys?

@tomscott anything new, dude?

I think this is perfect idea. I use XMPP in other app for notifications and is it very usefull for me.
The chat in diaspora is awsome idea.
But it must be standalone chat, no other server. Diaspora must respect privacy.

my suggest is keep it simple

a css/javascript client dropped into the template for logged in pages

and a place in the user profile/db where user can share his jabber id

i'm testing JWchat free code and have ejabberd installed to my pod, which works well if you want the jabber id the same as the d handle, but it should not be required to install an XMPP server, and the client should be able to use any xmpp server

+42 for @aj

I think two steps (diaspora accont and jabber id) will destroy the chat idea in diaspora.
Why? Users don't want communicate difficult. They want login & chat. No find or create jabber id etc. Only login and chat. Then i think define jabber in profile is bad idea, the better one is do it simple for user.

@maxsamael you need to read the discussion to understand why we want to do that.

@flaburgan thak you, i will check it

@max samael: Different users want different things. You can't say for all "users want only this" or "users want only that".

Sure, but what is point of decide? Who take the point?
I not, you not, nobody? Thats not true, someone take it. Than say what you want and I will say it too.
I don't say it must be there, but i say it can be there and i think im not alone. Then I say many users. Sometimes is update good and sometimes bad, but who decide about it? Say what you want and many people can listen and decide. I will decide too. And what do you think now? Are you free in speak. I sad yes, we are free and we wont be free. I think diaspora isn't there for north korean government, but for people. I sad for realtime chat YES.

@maxsamael Our decision-making process is based on the idea of rule-by-consensus. Basically, we get a feel from our developer and user community as to how we feel about any given idea: the technical implications, how a feature might work, what kind of work would need to be done, and so on and so forth. The point of this process is to see what ideas are:

1. Practical to put into motion
2. Have enough volunteers to work on them directly
3. Doesn't cause huge potential problems in our code
4. Is something that everyone voting wants.

Everyone has a chance to voice their opinions here. :)

@seantilleycommunit Agree, thank you

@maxsamael I recommend you read the discussion further down this page to see why XMPP was agreed on as the standard to use in a Diaspora chat feature. All the discussion is there to see. It was a long discussion and many different options were evaluated before XMPP was agreed as the best one for Diaspora!

Hi, this little post just to say that I agree with the idea of chat feature, and the choice of XMPP.

@goob Thank you, I will check it detaily

@tomscott @dennisschubert any news? Can we see your work somewhere?

@flaburgan there is no work since I had no contact with @tomscott about this topic so far...

As far as I understand it, a xmpp server component is more or less just a client that can act as a whole domain, so if we would use xmpp server component, we could not use a standard xmpp javascript client in the frontend, right? This would suck a bit if we would want to support non-diaspora jabber accounts later on.

Any news on this?

I think this is the next really wanted feature, with pod migration. @rogerbraun no news, if you want to give it a shot, you're more than welcome :)

I really want to get started with this, but I dont really see xmpp as a good first step. I think a more conservative approach of extending the existing messaging system would be a better and much easier start. This could be connected to xmpp with a server component later on.

@rogerbraun that won't ever yield anything near realtime communication.

I don't see why not. Messages can be displayed in a FB-like chat window and the client can be notified via websockets.

The latency issue isn't client <-> server but server <-> server

Ah, OK. I did not really think about the federation aspect...

Okay, how about this then:

A xmpp server is setup somewhere to handle the domain the pod runs on. On user account creation (or whenever, maybe after asking the users), the pod automatically creates an xmpp account and a password the user by registering with the pod via xep-0077(http://xmpp.org/extensions/xep-0077.html). Once a user is registered, all the pod does is give the necessary credentials to a client-side xmpp client (jappix or whatever).

What do you say? This would only involve writing the code to include a client and the code to do in-band registration.

We discussed that in the past and decided that this is not the way to go.

@dennisschubert I knew I did not come up with this on my own ;-) Sorry, the last time I read the discussion completely was some weeks ago, so I forgot parts of it.

Still, after re-reading the whole thread, I can not find any argument against this approach except that it might be hard to synchronize user lists. Is that all that would be bad about this solution?

As it appears from the silence that @tomscott is unable to work on this at the moment, would anyone else like to volunteer to work with @dennisschubert on it?

Of course if I understood the first thing about XMPP, I'd volunteer myself!

@goob it looks like @rogerbraun is interested to work on it ;)

Yeah, I'm still looking for an explanation why the xmpp-server + diaspora-server + automatic registration is a bad idea. It seems to be the easiest to implement.

@goob @rogerbraun I'd help you with the XMPP-parts, all you'd have to do is writing the modules :) If you want to, we can talk about it on #diaspora-dev later (in 3-4 hours) or you could add me on XMPP, [email protected] and I'll answer your questions as soon as I have some time left. :)

Just wanted to know something: is XMPP a secure protocol? I mean, is it encrypted communication?

If not, what about simply use IRC? I do that with jappix mini on my own website and it works pretty well.

You can encrypt data over XMPP, yeah?!

IRC is a groupchat protocol, you cannot send metadata, there are no contact lists or something similar. IRC is not designed for an application like we need, it's designed for very simple multi user chats, that's it.

Uhm, Jappix is a XMPP client not an IRC one.

I'd love to help, Dennis, but as Jonne can testify, any sort of coding is not something I'm good at!

@jonne I used jappix mini on my website with Freenode and it worked. But yeah, xmpp is definitely better.

MINI_GROUPCHATS = ["geexxx%[email protected]"];

That's an XMPP -> IRC gateway. It's still using XMPP.

@dennisschubert XMPP servers establish a TLS encrypted stream between one another for message transmission. Encryption can be negotiated and unencrypted connections can thus be refused.

@flaburgan erm, the previous comment was directed at you.

I think if we want to see progress on this, we should open a PR asap. It worked really well with the SPV, stuck for months, then @rogerbraun decide to start working on it and everyone followed. I think we could have the same approach here.

One big thing that keeps me from working on this is the current state of the frontend, with half of it using blueprint, half of it bootstrap, some parts are backbone, some not, etc... I am not sure that just dropping even more JS and CSS randomly into the page do much good. How self-contained is Jappix?

@rogerbraun we will abandon blueprint, so let's just work with bootstrap (and backbone if needed).

Has anyone taken into account the ability of XMPP servers like OpenFire to allow remote Server to Server sessions? This could act a little like Diaspora federation... Just a thought ;)

Is there, at the moment, any semi-simple way to share XMPP authentication with the local pod's user database? That is, the local pod user uses the same credentials as their diaspora* account to log into the XMPP client?

would a python script make for some suitable bridge to do this?

@jacobschleappi That's what XMPP gateways are used for. A gateway is a service that bridges between XMPP and some other protocol.

Hello everybody,

I'm new here so it will be my first contribution. I've read this conversation and I'll try to give a solution I imagine.

I agree with the fact that xmpp is best way to manage chat and offer a decentralized option.

So, we have the "xmpp server"(s), the "xmpp client" and maybe some data to synchronise or bridge between D* and xmpp databases.

I don't master Ruby at all but I work for it. I've read a little bit the code of the "Vines" project (https://github.com/negativecode/vines) that offers a xmpp server in a Ruby gem.

I think that with this project, run the xmpp service from the D* database would be pretty easy and "integrated" . It "only" need to hack the storage managment of the "Vines" project to add support to the D* database (users/people/contacts/aspects). It currently uses Nokogiri (and optionaly LDAP for auth) and have a gem that provide the support for storing in sql database. (https://github.com/negativecode/vines-sql)

With this solution, no need of 2 differents accounts on D* and xmpp and the ID in both nodes are automaticaly the same. No need of two databases. The Pod offers the xmpp service directly from the D* database if these gems are installed on the server. Users that host their account on this Pod have access to their new xmpp account in the « Jappix Mini for D* » (for example) or they can connect from another xmpp client to their D*/xmpp account. Their contacts and groups are the same. The D* node become a xmpp node too.

That's a good idea, but it's also heavier than what we tried to be. Linking an external XMPP server is way lighter than integrate a XMPP server inside diaspora*. It would give the choice to podmins to install the server on the same machine, another one, or simply redirect users to an existing XMPP server (jabber, duckduckgo...)

Using an external service requires further action on the part of the user. The whole idea was to make the transition as simple as possible. Jabber (Openfire as an example) only uses minimal memory. I imagine Vines (the Ruby gem) would use only a small amount as well. As for processing power, its not very intensive at all.
I don't think pushing a user to an external service is ideal. We want to keep the pods independent, but allow for federation. Per pod XMPP would make it easy. Both the pod and XMPP server can use the same SSL cert with minimal configuration.
TBH - I think its time to make the choice.
Heroku was a good place to start while there was little funding and only startup pods.
However, there are much more comprehensive solutions now that are funded by people dedicated to the cause. @davidmorley and myself are examples.
I fund my pod out of my own pocket. I don't ask for donations, and I only pay for a server I can already afford.
If the likes of Joindiaspora are on a single Heroku Gear, then I'm pretty certain that its time to upgrade. With the amount of traffic that they have and all.
The Diaspora Project cannot afford to be held back by those who are unwilling or unable to move forward with the times.
Yes, Joindiaspora is the largest of the pods, but it is not the main face of Diaspora. The two biggest places for information regarding D* are the diasporafederation page and podupti.me
Remembering full well that @davidmorley hosts podupti.me himself.
Yes, we may lose a few pods due to funding issues, but what we will gain is momentum. Something this project is sorely lacking in.
If you want to keep Heroku in the loop, then the only option is to code around the limitations of the single Gear. Make the XMPP server an optional install, like a module.
Having an extra few steps on the wiki about how to include something like Vines in your installation wouldn't hurt anyone.
Joindiaspora will still be able to federate, but without XMPP chat. If they want it, then they can upgrade their server to a paid or sponsored system. Otherwise, it would be no different to what it is at the moment.
No harm, no foul. Everybody wins.

@starblessed we already voted about that and we decided to not be stuck by Heroku. What I had in mind saying we have to keep diaspora as light as possible was more about someone wanting to host himself on a raspberry PI for example.

I just wanted to say that, in my mind, having a chat in diaspora became number one in the TODO list. With the NSA scandal, I see more and more people coming on my pod. We are freeing them from facebook and that's awesome. But there is something worse than facebook: google, and especially gmail. To have another address than gmail is easy, but many people doesn't change because of gtalk. Being able to propose an alternative which, besides, allow them to talk with gtalk contacts is awesome. We have to include XMPP chat in diaspora as soon as possible, it will help the XMPP network to grow and become more known outside the geek sphere.

@dennisschubert @tomscott @rogerbraun @sakwe and @vittoriocuculo too if you want to help, I'm sure that if we work all together, the dream can become true! Let's do it!

About the chat, I can propose my time and knownledges to add the support of WebRTC video/audio.
This is currently only supported inside recent Google Chrome/Chromium and Mozilla Firefox versions.

And if necessary, I can also add a fallback support for others browsers to uses Red5 and Adobe Flash plugin ( but this is a bad idea IMHO ).

@guillaumelakano: What is the current state of Jingle over WebRTC support? It was still in development a while ago. Is it already usable to be added to Diaspora chat?

@shmerl : We haven't to understood what library is used by the browser ( libjingle indeed ), it's all transparent for the developers.

It's already useable yes, I have already done a project for a client with WebRTC. This works between Firefox and Chrome browsers like a charm ! This also works on my smartphone Android with Google Chrome updated.

May be I haven't see it, but currently I don't see any chat available on Diaspora (I'm under diasp.org )

@guillaumelakano: It’s not about libjingle, but about support for the XMPP/Jingle protocol over WebRTC.

I.e. imagine such use case:

1. User A logs in Diaspora chat, and sees contact B there. Contact B is logged in through standalone XMPP client (Pidgin, KDE Telepathy, Jitsi or whatever).

2. User A initiates a video/audio call, which goes through WebRTC and uses Jingle protocol for signalling. User B picks up the call and they communicate.

In order for that to work, Jingle implementation through WebRTC should be already in a workable state, that’s what I was asking about. I have no idea where things stand now.

I would like to say that I think the lack of chat in diaspora prevents many people from using it. Also I have noticed an increasing number of people registering on my pod lately. As I see it, it is time to support a secure chat.

WebRTC looks great but if XMPP is more secure I would prefer XMPP.

@emmanouelkapernaro: As discussed above, WebRTC is not a substitute for XMPP, they are different level frameworks.

@shmerl : I don't understood why you ask questions about Jingle. There is nothing to known or to understood about Jingle, it's automatic, as the video codec.

I explain how this could works in Diaspora :

1. User A - Want to start the camera of User B, he click on the chat link "Ask camera of User B".

2- The chat ask authorisation to User B. If B accept, the chat send a special random unique identifier to User A, and start the WebRTC connection

3- User A receive the random unique identifier and start the WebRTC and attach the connection to this identifier.

That's all !

Nothing to worry about codec or jingle !

And with this technique, we can also permit for a external XMPP program to see the camera with a http link.

@emmanouelkapernaro : XMPP is for the chat message exchanges. WebRTC is a new HTML5 technologie to permit to exchange binary stream in P2P as the camera/audio.

@guillaumelakano: It looks like you are missing some basic facts there. WebRTC does not mandate signalling protocol which applications use to perform media calls. In order to have proper chat with media capabilities, you need the whole stack. Transport layer, presence protocol and signalling protocol. WebRTC is essentially the transport layer. It does not mandate what presence and signalling protocols are used. In XMPP world, presence protocol is the XMPP itself, while signalling protocol is called Jingle (you can read about it). There are other signalling protocols like SIP for example. In order for any client to have any clue how to communicate you need to use some of these protocols, otherwise you'll have zero interoperability.

From the WebRTC FAQ: http://www.webrtc.org/faq#TOC-Why-should-I-use-WebRTC-

WebRTC abstracts signaling by offering a signaling state machine that maps directly to PeerConnection. Web developers can therefore choose the protocol of choice for their usage scenario (for example, but not limited to: SIP, XMPP/Jingle, etc...)

I hope this clarifies things. So, let me ask this again. How does implementation of Jingle over WebRTC fares now, is it even ready?

So let's go back to my use case but from different side. User A related to Diaspora opens Pidgin (or Jitisi or etc.) and logs in to the XMPP server. Some contact B of that user is using Diaspora web client at the same time, and they see each other as on line. User A (in Pidgin) initiates video call to user B. Pidgin makes XMPP/Jingle call to user B as proper. The client of user B (inside Diaspora site) uses WebRTC to handle all that. So how will that work exactly?

From your example, it looks like you are trying to make your own makeshift signalling protocol. It's not an acceptable approach since you won't be able to communicate with any other client (besides those which use your custom non standard signalling). That's why using Jingle is important.

@shmerl : Ok, now I understood better your questions.

In my development cases, we only used HTML clients, so we haven't the same problematics.

The audio/video is available between 2 users inside the diaspora web client.

I have no idea about when the WebRTC will support Jingle or SIP capabilities to permit to choose directly the internal protocol to be compatible with external clients as Pidgin.

I said in a previous message "with this technique, we can also permit for a external XMPP program to see the camera with a http link.", it's a technique by opening a popup browser, and not by using the internal support of Jingle of Pidgin

Yeah, I see a serious downside in implementing a custom non standard signalling to use with D* web-client <-> D* web-client only and which won't be compatible with any other (non D*) web client or standalone XMPP client for media calls. The proper way to do it - is to implement Jingle over WebRTC when that will be possible. If you can look into that in the context of D* chat - that would be awesome. Thanks.

@shmerl : In a first step, I think we can make a video/audio only between D* web-clients.

And when this could be possible, we'll upgrade to Jingle to have interoperability with other XMPP applications.

After some search, I have found a not yet working solution but this should automatically works in future.

There is 2 library ( stanza.io and strophe.jingle ) that permit to uses Jingle protocol inside WebRTC A/V calls.

But Philipp Hancke, the developer of strophe.jingle, explained me it's partially working in Chrome 28, but not anymore in 30.

He advice me to uses the stanza.io library, even if this not working fine today, this should be fixed in future versions of Jitsi / Chrome.

The discussion is here :
https://github.com/ESTOS/strophe.jingle/issues/32

Thanks for the update!

so... speaking as the developer of the strophe.jingle stuff and also contribute to the jingle stuff in stanza.io.

WebRTC is bring-your-own-signalling. It's just a media engine which captures data from your webcam or microphone and sends it to a peer. How that is negotiated is left to the javascript. Using Jingle, which is XMPPs way of doing this is one alternative. Check out http://vimeo.com/77289728 for a great introduction to webrtc and why you should avoid building silos (little chance here :-) and use XMPP.

The biggest advantage in using a standard like XMPPs Jingle for WebRTC signalling is of course interoperability. Otherwise you could just buy the "just use our API and exchange an URL and be happy" story that some people are selling in that area.

So strophe.jingle currently kind-of works with jitsi. stanza.io doesn't yet. That is going to be fixed at some point, but fixing and stabilizing interop between strophe.jingle in chrome30 and jitsi is currently a higher priority. In the meantime, XMPP offers a way to detect that a client is not capable of doing WebRTC+Jingle and fall back to sending a URL. Now Jitsi is one of the audio/video clients that drive development of modern protocols. Others, like pidgin, are still lagging behind. WebRTC doesn't allow unencrypted media traffic at all.

There is actually a third jingle implementation in the browser at https://github.com/legastero/jingle-interop-demos and I am expecting another one to join us very soon.
What library you choose is mostly a matter of preference of the guy who develops this. stanza.io is a quite good approach to hide the uglyness of XML (aka "legacy JSON") from web developers by letting them work with JSON objects.
strophejs is an dependable workhorse and there is even a book explaining it. But it's age is starting to show.

Thank you Philip for these precisions. I think that we should start creating a prototype now, what about an IRC meeting to decide who wants to do what? When are you guys available?

@philipphancke Thanks for sharing your experiences :)
@flaburgan IRC will be difficult to match a convenient hour for everyone (we are not in the same GMT)

I propose to see here, who want to works on the chat.
As explained before, I could give time to add support of video.

About the XMPP chat, I don't known if someone have started something, and how we could attach it to Diaspora website.

About the XMPP chat, we don't need to re-invent the wheel, there is a library that permit to do that easily :
http://conversejs.org/

When the XMPP chat was integrated, I (or any other volonteer) could add the video support with one of the 3 libraries mentioned above.

Others, like pidgin, are still lagging behind. WebRTC doesn’t allow unencrypted media traffic at all.

@philipphancke: What kind of encryption is mandatory? I guess telepathy based clients (KDE Telepathy, SailfishOS XMPP client and so on) won’t be able to interoperate with WebRTC based JavaScript XMPP/Jingle client as well? Telepathy still lacks end-to-end encryption support.

• https://bugs.freedesktop.org/show_bug.cgi?id=29904

• https://bugs.freedesktop.org/show_bug.cgi?id=16891

Hi every1

I'm the author of https://conversejs.org which @guillaumelakano mentions two posts above me.

I think converse.js would be a good fit for Diaspora and one of my goals is to make it easy for people to integrate it into distributed FOSS projects such as this one.

I'd be happy to answer any questions or help out how I can if you guys decide to test-drive or use converse.js.

For the privacy minded, I recently added Off-the-record encryption support. [0]

I intend to add support for video chat, most likely using @philipphancke's strophe.jingle[1] library.

If anyone is interested in contributing, please let me know. Currently I'm working on this alone and in my free time.

[0] https://opkode.com/media/blog/2013/11/11/conversejs-otr-support/
[1] https://github.com/ESTOS/strophe.jingle

@jcbrand It's good to see you on this topic ! :-)

If you include strophe.jingle of @philipphancke into your converse.js, it's will be perfect to add the XMPP/video chat on Diaspora !

About the XMPP chat for Diaspora, we need to discuss about how to deploy it. Do we request to each pod to install an XMPP server ?

I think this could be usefull if D* users have a default account set on the pod AND have the possibility to change it to set their own XMPP account

It's good to see you, JC, and thanks for taking the time to join this discussion.

Guillaume, on your last point, perhaps as you say users on pods which join the chat service have a default chat account, ideally set to be the same as their Diaspora ID ([email protected]). There is then a setting in user preferences 'Use an existing XMPP account'. If the user enters an existing account name in here, they can use that account in Diaspora, and any calls to their Diaspora ID get forwarded to this account, so someone can contact them via their Diaspora ID even if they don't know their XMPP account name. Does that sound like a good idea?

Thanks @guillaumelakano and @goob

@guillaumelakano I was playing with strophe.jingle about a month or so ago and noticed that it is tested with Prosody XMPP server and doesn't yet work with Ejabberd (which is what I'm using ATM). I'm sure Ejabberd and other servers will eventually catch up, but this is something to keep in mind. This is bleeding edge stuff and broad support among browsers (for WebRTC) and XMPP servers is still lacking.

About deploying XMPP servers, I think we could perhaps do this organically. Can we not let pod owners decide for themselves whether they want to install an XMPP server or not?

There are public XMPP servers available (like jabber.org) and many people and companies host their own XMPP servers already. So certain people might want to re-use those accounts.

I can think of mainly two ways of adding XMPP support (both don't require an XMPP server per pod).

• Tight coupling of XMPP credentials and Diaspora user account
• Loose coupling of XMPP credentials and Diaspora user account.

Starting with tight coupling:

The important thing is that the user give their Jabber credentials to the Diaspora pod, and the pod then keeps a mapping of the user's Diaspora ID and Jabber ID and manages the users chat rosters.

If you "friend" someone, you'll automatically be able to chat with them (i.e. they'll be on your XMPP contacts roster) and there would be no need to add them again via Converse.js just to chat.

So, when you accept someone's friend request, the pod looks up that person's XMPP username and sends an XMPP contact subscription (on your behalf) to that persons. That way the new "friend" automatically also becomes a chat buddy.

This will however require server-side code that can talk XMPP and therefore take a lot of time to implement nicely.

And then loose coupling:

To quickly add XMPP functionality, you could let users manage their chat buddies via Converse.js. In this case, the pod doesn't need any new code, you don't necessarily have a 1-to-1 mapping between Diaspora users and chat users and when "friending" someone they don't automatically become a chat buddy.

It's not so tightly integrated, which sucks a bit, but it can be implemented relatively quickly (and the other functionality can then still be added in the meantime).

@jcbrand About the choice for the admin to install or not an XMPP server, I think it's a "wrong good idea", because if only few pods install an XMPP server, we need a fallback (for example, accounts created on the default diaspora XMPP server) and this mean "centralized" ... it's never a good idea to centralize the majority of all accounts on one server.

Or... we can keep it as an option for the pod admin, but we really need to insist about the necessary to install it on their server...

About the way you describe for adding contact, to resume, when the web client D* receive the request :
- We search if this XMPP account is attached to a D* user.

• No:
  We ask the question and precise it's not attached to a D* account (may be an option "ignore external" accounts could be added)

• Yes :

  if I follow him, we automatically accept him, to permit to chat directly.
  If I don't follow him, we ask to the D* user if we accept his request with the D* name + link of this member.

Is this possible to make XMPP server a part of every pod and share contacts, domain and usernames between pod and XMPP server automatically? Is this reasonable? This could make some things easier for pod admins, which is important because we want to have thousands small pods rather than few mega-mods if I recall properly.

If you review the discussion above, you'll see that general consensus was about some kind of loose coupling of D* account / server with XMPP account / server. The main reason for that is flexibility and also the fact that some pods run on Heroku which doesn't allow running any other servers (like Ejabberd or the like). Therefore it should be up to podmins to decide whether to deploy XMPP server together with their pod or not.

But I don't mean server like ejabberd which is a separate application, rather my idea is to write XMPP server in ruby as a part of diaspora pod application. What about that? Would heroku allow app to listen on many ports?

I don't think you want to write your own XMPP server (let alone in Ruby) and keep it feature rich, unless you have time and resources for that. It's not a small feat.

Allowing using any external XMPP server (which can run on the same host when possible) looks like the most practical solution.

@macieklozinski Well, your idea is interesting, but if we start to code an XMPP server in Ruby, this will take a lots of time.

@shmerl I think we doesn't have to take decision dependant of what Heroku propose. Of course, I understood some of podmins uses Heroku, but all podmins are not under Heroky fortunately ;-)

The idea of Maciek Łoziński to include an XMPP Server in Ruby is may be a way to discuss. I have found Vines ( http://www.getvines.org/ ), an XMPP server in Ruby.

@shmerl If we add a ruby XMPP server inside the code, do you known if it's help Heroku users ? Could they launch Diaspora AND Vines at the same time on their Heroku ?

If yes, we need to see with @philipphancke and @jcbrand if the Vines server is enough to support all features required for the chat / video.

This could be set as a default server but with the possibility to uses another server ( ejabberd, prosody ...)

I'm not familiar with Heroku, just know it's limited to Ruby on Rails and it's not a general purpose hosting, so let others comment on that. Writing Diaspora own custom XMPP server looks like a waste of resources to me, but of course anyone is free to do that if they want to.

@shmerl The idea to create a custom XMPP seems a waste of ressources for me too, but it's why I propose to include an XMPP server coded in Ruby, open-source, Vines ( http://www.getvines.org/ )

It's not a question of what language the server is in, it's the question of what Heroku allows you to host. I think Ruby on Rails is not conductive for XMPP servers.

There has already been a vote about the issue of Heroku, and the decision was that we should not limit our ourselves to finding a solution which works with (free) Heroku. See https://www.loomio.org/discussions/3678?proposal=2643

Any solution needs to be compatible with a network which contains pods hosted on free Heroku, but the chat solution does not have to be one which can be installed and run on a free Heroku account.

@guillaumelakano @sakwe already proposed vines a few days (weeks) ago. We are looping in our discussion guys ^{^}

So to summarize, as goob said:

• Heroku support is better but not necessary
• What we want first is a chat. Video would be nice, but a simple XMPP chat would already be good.

Now if someone (@sakwe ?) could code a simple proof of concept using vines to see if it's light or not, and if it works on heroku, that would be nice.

@Goob Diaspora on free Heroku account with the current federation system will not work, because of the replication of posts the db with max. 10k rows is the end. Otherwise the user needs to pay. We should not focus on Heroku.

Vines sounds good - first we need a proof of concept. Let's start coding :-)

@diaspeu

We should not focus on Heroku.

That's what I'm saying!

It would be odd for a free software project to focus on making sure hosting works with one commercial provider.. ;)

So, if you do not need to worry about Heroku, we can simply ask to podmins to install an XMPP server of their choice, it's simple :)

1) So, we use the same D* ID for the XMPP ?
I mean, my D* ID is [email protected] then, this will be also my XMPP account, isn't it ?

2) if a podmin doesn't want or can't install an XMPP server, do we plan an fallback ?
I propose to make this simple, and to not make fallback, because fallback mean a centralized XMPP server (and it's never a good idea)
So, if users want chat services, they need to change to a pod with chat support.

3) We also need to discuss about the possibility to change pod. What happends to our previous XMPP account.

I think it would be enough to implement a solution which allows pods which have an associated XMPP server to communicate with 'chat' with other pods which have an associated XMPP server. Not every pod in the D* network would have to participate in this chat solution.

Users who wanted to participate in chat could move to a pod which provided the facility. Podmins who didn't have the resources to run an XMPP server needn't be under pressure to try to install one in order to continue their pod within the D* network.

Yes, I hope finding some time to work on a little code I have proposed with Vines. I confirm that Vines works as an independant XMPP server. I have installed this from the Ruby gem. BUT I'm not a Ruby programmer so, I have to learn a little bit about it...

The code I propose to write is an adapted code of this gem (https://github.com/negativecode/vines-sql) that enable Vines XMPP server to use a SQL database instead of the master storage managment.

I think that it could be enough to make the Vines XMPP server totally integrated to the D* database.

So we could imagine that a Podmin would have to install the Vines server and our "Vines-D*-stograge-adapter" gem with the bundle and all would be ok for the server part of the chat capability.

BUT with this solution, the XMPP server is not an external service and it become a part of the node Pod (the D* node become a XMPP node).

Personnaly, I prefer this because final users don't have to create another XMPP acocunt to chat on D*. Their D* account is their XMPP account. Their D* aspects are their XMPP groups, their D* contacts are their XMMP contacts.

The SSL certificate is the same for both services...

PS : I have a question if someone can answer me... I'm making tests on Jappix Mini integration with a "pre-login" (login/pass from a database). Is there another way to auto-login Jappix Mini than with the "launchMini()" function ? Or if no other way, is it possible to crypt the password we ha to feed in the function call ?

Personally I think we should make pods lighter, not heavier. It's already quite resource heavy and people want to run pods on light stuff like Pi's or even routers. We're not going in the right direction if we bundle more functionality in.

But I guess with proper planning components could be opt-in and thus keep the core light and the extra weight as optional.

Sure it is an important question. I can say that the XMPP service is very light so I don't consider it as a real overload for the machine.

As Vines is a Ruby gem, a podmin that can install D* on a server can install the XMPP server. It only need some openned ports on the server.

I think that the best solution for D* users is to have automatically access to the chat capabilities with the same ID for both services.

Asking an user to add separeted accound with no link with his D* datas (essentially the id) is not a real integrated chat. Most of users will not unedersand this, only "power users" will use it I think... No?

Sure it needs to be integrated, agree totally there :) But that can also be done via provisioning. But didn't really want to interfere on the XMPP discussion, no knowledge here, just saying it's not necessarily always a good idea to bundle everything together into one component :)

We should learn from our failures :-) 28 changed files with 12,708 additions and 3 deletions without any result:

https://github.com/diaspora/diaspora/pull/1750

In Vittorios approach we used ejabberd. Vines sounds better because of the seamless diaspora integration.

I think the thing to learn from that project is 'don't develop anything for a project whose core developers are making enormous changes to the code base without communicating this'. This is no longer the case! And once more modularity is built in, changes to the core code base will be even less of a concern.

I think D* shouldn't mandate the XMPP server. It can be Vines, Ejabberd, Openfire - whatever the podmin likes more. D* should be decoupled from the XMPP server. If podmin can't install it (Heroku, etc.) - then that pod won't have chat feature. Alternatively, the UI can allow connecting to external (hosted elsewhere) XMPP server.

Decoupling won't tie D* to the limitations of the chosen server, and if some XMPP server will get ahead in features or development, podmins will be able to easily switch. Hard tying D* to one server will make that impossible.

If we want to have an integrated chat functional that offer the possibility to be decoupled from the pod, it mean that IDs/passwords are not the sames on both services. It means that databases for both services will not be the sames. It means that the end-user have to understand the difference between D* and chat in D*.

In this way, if we want an "integrated" service, we have to code functions that will synchronise (in the best way) the D* contacts and aspects with the XMPP contacts and groups. I think it can be done by BOSH communation.

So, this solution could exits but it means I think a lot of new code in the D* core to send, receive and confirm BOSH messages with the xmpp service.

Example: A new "aspect" is created, we have to send a BOSH request to create the same "group" in the xmpp account. (same for contacts, add, delete, move, ...)

I think that we loose a part of Pods federation with this solution too...

Example : If "[email protected]" want to communicate with "[email protected]" on the D* network, it's ok, the federation is done. But if "[email protected]" have an xmpp account "[email protected]" and "[email protected]" have "[email protected]" as xmpp ID...

How does the federation could help us without a new hard functionallity in D* and a global update for all Pods?

And still the same question. How many users will undestand that they have a D* id/pass for D* but a different xmpp id/pass to chat ?

I add this reflexion... I think that decentralisation have limits : the user. It's not "user friendly" to decentralise a user.

I believe there is no need for a user to use some preexistent xmpp id. If he has already an xmpp id, he probably uses it with pidgin or some other app. He can use his new diaspora xmpp id with pidgin too.

So I think it is better not to have the option to import an external xmpp id into diaspsora. Every diaspora id will be automatically an xmpp id too.

Coupling of the server with D* has more problems than benefits which a unified ID gives. You can make enough integration with a flexible XMPP server decoupling.

Example:

1. User creates an account on a D* pod which has associated XMPP server configured. Under the hood, account is created on that XMPP server which matches D* handle, and with the same password. When user changes D* password, XMPP password is changed accordingly.
2. User is provided an option in UI, to bind these IDs, or to unbind them (by default - they are bound). When bound, D* will use the matching JID, when unbound, user can set explicit XMPP id on any other server.

This allows several things:

1. No need to require Vines only. Any XMPP server will do, as far as there is a unified way of creating a user there and setting / changing passwords.

2. It gives an option for "power" users to use other XMPP handles besides the default one.

Handling authentication for presence and contacts can work similarly. Not everyone I add on D* in my aspects should be authenticated for my XMPP presence, and the other way around too. User should be given a choice, since those aren’t necessarily always matching 1:1.

I believe that it would be more user friendly when users are able to use their existing XMPP accounts from whatever server they want. Many of these users probably already have accounts with facebook and google to add in their Pidgin and Jitsi clients. Avoid the confusion of having two additional unchained XMPP accounts and having to ask all their contacts to add yet another ID for them.

It wouldn't surprise me if most podmins are already running an XMPP server and would like to continue only running one.

As @emmanouelkapernaro said, it's surely better to not uses an existant XMPP account for users.

I propose to make it simple to report the feature of using an external account for later.

Like that, it's better to concentrate only on 1 account created on the fly on the pod, with the same D* ID

Yeah, I also think that using the existing handle as the xmpp id is the simplest and less confusing way to go. @sakwe I encourage you to work on a proof of concept and ping DenSchub, MrZYX or Raven24 on #diaspora-dev if you need help about Ruby.

I think we should let @sakwe continue with Vines integration. If anyone feels strongly the XMPP server should be decoupled, they can implement that too. It can be an option for the podmin to choose. The important part here is to get chat working as soon as possible. If someone is working on code, lets not block them with tougher requirements.

As I see the "Vines solution" seems credible, it motivates me to write this code. Unfortunately, I am still very busy this December to devote myself. In January, I will work on it.

What are the tasks for the Vines integration?

@diasp_eu The first code to write is an adapted code of this gem (https://github.com/negativecode/vines-sql) that enable Vines XMPP server to use a SQL database instead of the master storage managment.

I think that it could be enough to replace Vines data queries into its local database to queries into D* database.

Happy new year to everybody!

@sakwe thank you for helping to the XMPP integration, when could you start working on it?

I comment because I don't want we forget this excellent post on Loomio with the christmas holidays.

Unfortunatly, I need to work on another project for the moment and it take all my time. But I don't forget this functionnality and I'll have time soon to work on it.

Ok, thank you anyway and good luck!! :)

Hi guys.
I image this is the right place to know about, so how is going the chat development?
Is it going to be XMPP based?
I know we need to be patient about that but I asked because there are many people asking me all time and I have no answer to give them. Tks

I certainly hope we don't stray away from the XMPP plan, it seems that is what most of us want currently and would be ideal.

We still need a developer that has time for this.

Cool, the XMPP is indeed an amazing choice.
I'm gonna try to invite some developers I know, let's see what happens. Unfortunately I can't help much on dev.
Thank you @sethmartin

Hi, I have done some work in the past year to integrate a sitewide XMPP chat system for the company that I work for. The application itself uses rails, for an XMPP server we chose Openfire plus the Punjab BOSH server, and we wrote a JavaScript client on top of Strophe. While I am by no means an expert on all of the intricacies of XMPP, I feel that I could offer some assistance. I believe we've successfully served 10k+ users (being conservative but we stress tested a lot higher than that) online at once for anyone concerned with performance.

I agree with some of the previous comments that the XMPP server shouldn't matter and could be left up to the podmins. A JavaScript XMPP chat client can be written to work with any BOSH server. For those that don't know, BOSH wraps XMPP in HTTP.

I see that the other alternative is Vines, which I haven't used so I don't have an opinion. If that is the better choice, by all means go for it.

WDYT?

As an addendum to my previous post, figuring out how to authenticate users on a standalone XMPP is an interesting problem that would likely require extensions to each server that a podmin wanted to use.

@davidthompson awesome that you are offering your help here - a lot of users really want chat but we just lack the hands to do that - so help would be appreciated!

There has been a lot of talk about different options but really no discussions. I think the way forward would be to make a concrete suggestion of technical detail (maybe in our proposals page in the wiki, or just here briefly) - and then we either accept it, modify it or reject it, by voting if necessary. In the end, the technical solution should strongly be influenced by the person who is actually willing to do the work ;)

Sorry, "no discussions" = I mean no concrete full technical proposals that could be accepted :) There has been a lot of discussion of course.

Relying on XMPP as the basis of a Diaspora* chat is a great idea, but we have to consider that plain XMPP isn't really usable on mobile devices. So when thinking about which server to use or which featureset to implement please consider that it should work great with mobile devices.

This article describes why the current state of most XMPP clients and servers sucks for mobile use: http://fnanp.in-ulm.de/blog/2014/01/16/01-woes.html

We have the chance to do things right from the beginning!

I have Xabber on my phone and it works great - though I just idle there since I don't really know anyone through there lol. Would be great to be able to chat with d* guys with a mobile client.

@jasonrobinson according to the blogpost Xabber is one of the clients which don't support the important XEP-0198 which sometimes causes a silent loss of messages. And this is what I mean

I use Jitsi on my mobile, it works just fine here. Including voice and video (read Jingle).

@davidthompson maybe you can see with @dennisschubert and @sakwe who proposed their help too.

I wouldn't want any XMPP solution that didn't permit my using Prosody for the server. Prosody has an XEP-0198 extension which is necessary to be reliable for mobile users as @trollischmittlauch points out. I wouldn't want to prevent any podmin from choosing their own though.

As for mobile clients, ChatSecure is the real winner at this moment. I was using Xabber which was very nice but after discovering that XEP-0198 support was an absolute must for mobile, ChatSecure was the obvious choice. The article that @trollischmittlauch links to above, claims this client to have connection issues but I haven't experienced this on my device.

I'll say again that you can use a server component with any XMPP server out there. If the server component implements XEP-0100 (easy to implement, see Libertree code) then a user can connect any JID (even on remote servers) with the Diaspora account. If you don't like that an existing JID is required to use it with a regular XMPP client you can transparently create an account on a local XMPP server (even with the exact same name as the user handle), because there's a XEP for that.

Integrating a full XMPP server in D* is against the spirit of XMPP and is a terrible idea from a software development perspective.

What to use on the frontend-side is a different matter.

@trollischmittlauch "silent loss of messages" is always a possibility with message stanzas. Iq stanzas, on the other hand, have to be acknowledged by the receiver and are retried if there is no response.

I can make the implementation for Pidgin, Instantbird and Thunderbird with libpurple protocol plugins.
We are waiting for deploy the service.

@alexandersalas what do you mean by "make the implementation" and "waiting to deploy the service"?

@jason-robinson Pidgin need plugins [1] for new protocols (example: prpl-diaspora) with libpurple [2], i can make the implementation of this new protocol for Diaspora, i make in the past another addons for Instantbird for example [3]

[1] https://developer.pidgin.im/wiki/ThirdPartyPlugins#AdditionalProtocols
[2] https://developer.pidgin.im/wiki/WhatIsLibpurple
[3] https://addons.instantbird.org/en-US/instantbird/addon/358

Thanx for your offer!

We definitely don't want to invent a new format for chatting. We will be adapting XMPP and so pidgin will be able to connect with diaspora out of the box. So I'm not sure if there is anything to be done for pidgin at all. Maybe extending the menu for a diaspora user to easily access the profile of that user from pidgin or something like that?

@rasmusfuhse If Diaspora behaves as a regular gateway service the client only needs to implement the gateway workflow to allow for chat with Diaspora users. Pidgin and Psi are just two of many XMPP clients supporting service discovery and gateway interactions.

After 11 month of discussion we should vote for a technology and go for it.

In this meanwhile, does anybody know what happened to this project below?
http://blog.ninux.org/2011/08/30/gsoc-2011-videochat-for-diaspora/

The author of this blog-article is @vittoriocuculo who also brought in some comments here. Back at the time there were automated tests missing, so his pull-request was rejected. Right now his code is unmergable and Vittorio doen't have so much time anymore. But his work can be used to reimplement the chat and his experience will be helpful.

So, we fixed XMPP for chat, or should a new decision proposal to be opened? And, if it has been fixed, what is the coding status?
Sorry for being impatient (and that too without help in the coding side, but when it comes to Diaspora, I am a mere user and evangelist, and never a coder. Don't know Ruby or Rails. :( ), but when I speak to people about D*, they ask me about chat and I really don't have anything to say.

@balasankarchelamat we agree the way it should be done, but nobody started to work.

Hey guys, maybe some of you already know about that but there's an XMPP API code called Jappix Mini.
I'm not a developer and I'm also not sure if this could help somehow for a start, but the javascript code is in the link below, just select option 4 (Installation):
https://mini.jappix.com/get#

@rasmusfuhse , thanks for the answer.

I've started messing around with the jquery xmpp plugin, which is BOSH based. The first thing I ran into was the need to conflate the user's Diaspora contacts' JIDs into the user's XMPP account's roster (see https://diaspora.koehn.com/posts/148942). It looks like you're thinking that the pod should have access to the user's XMPP account, which is pretty much the only way for this to work. There's a lot of server-side coordination to keep the two accounts in sync, which is complicated somewhat if the user logs in with another XMPP client and starts to add/remove JIDs themselves.

Assuming we can solve this problem, the next problem I see is that if you have a widget on screen into which you're chatting, the widget will disappear every time you switch pages. If the widget is doing the XMPP/BOSH stuff itself (which probably makes the most sense) then the user will lose their chat state (history, partially-composed IMs) every time they switch pages as they look at individual posts, change aspects, etc. since all of those involve changing pages.

We'll need to make Diaspora into a one page application to keep the widget visible as the user navigates the pod, which is what Facebook and Google+ do. That's a significant chunk of work, I'd imagine. You might be able to hack something together with frames, but everyone hates frames for good reason. A last option would be to have a separate window for chat, but that has negative implications for mobile.

BTW My Jabber ID is [email protected]. :-)

A diaspora user suggested checking out converse.js; it does most everything we'd need.

Awesome @bradkoehn :)

Personally I'd be happy initially to just integrate XMPP provisioning only to start with. This way podmins can run a server themselves and auto-provision accounts for their users. For contacts, xmpp ID would show in the profile.

Then second step - integrate chat into UI.

Any suggestions on how to do this? I imagine it's partly a matter of configuration: the domain of the pod should match that of the XMPP server and an admin account must exist on the XMPP server which the pod can be configured to use to create XMPP users with the same password used in Diaspora. A secondary requirement is that the XMPP server should support BOSH and have the appropriate SRV and TXT records to make it easy to communicate with.

Once that's done we need to add the user's XMPP id to their user record and begin transmitting it to other pods. I think that that meets the requirements of Jason's first step.

What else needs to be done? And how do I set up an RoR IDE?

@bradkoehn Have you evaluated the XMPP server component approach yet? Chat can be handled very easily within a server component as you are not restricted by XMPP stanzas. That's what we do in Libertree --- there is no additional requirement on the XMPP server side. Everything is handled by the component, allowing for a minimalist chat implementation. No BOSH needed. The client doesn't have to speak XMPP at all, that's the job of the component. This also makes it possible to have chat without XMPP (at a small performance cost).

That's certainly another option. It would seem to put forth some new requirements for developers (mostly that a chat component be available for Ruby to act as a gateway) while eliminating some complexity for podmins (setting up an XMPP service is onerous, as I just discovered getting ejabberd running on my host). It's certainly worth considering.

@rekado How did you handle the problem of switching pages in Libertree? Or did you? converse.js does its best (I've been trying the roundcube plugin), but the user still loses information when switching pages. Migrating Diaspora to a one page app à la Google+ or Facebook would be a massive undertaking, but it's required to get chat working correctly, especially if one adds voice/video.

setting up an XMPP service is onerous

My experience with prosody has been the opposite. It was less work than setting up an HTTP server.

How did you handle the problem of switching pages in Libertree?

I don't know why you think that would be a problem at all. Every chat message passes through the XMPP component by design. Hence it can be persisted and replayed to the user on demand. We simply store recent chat messages in the database.

We don't use converse.js because it's not needed. We only need to speak XMPP in communications between servers, not between backend and frontend. In Libertree chat messages received by the backend are stored in the database and pushed through to the frontend via websockets. Really simple. We are not dealing with stanzas in the frontend, because that is not required to make chat work. We don't need a full XMPP client in the frontend thanks to the privileged position that an XMPP server component has --- it can author stanzas and the XMPP server transmits them. We have no need for any of the usual tasks of an XMPP client because we only need to talk to the XMPP server component using whatever means makes sense (in our case: through the database and websockets).

A chat message sent off in the frontend has to go through the XMPP server component as well. We never lose conversation state and do not need to implement all that in JavaScript either. You are welcome to copy chunks of our code if you find that helpful. See the libertree-backend-rb repo.

I guess the real question we need to ask with this approach is, whether it scales to a few thousand concurrent users that some Diaspora pods have.

@rekado Thanks; I hadn't thought it through that far. So you must be sending keystrokes up to the server, and thus maintain a lot of state server-side (e.g., who you're chatting with, messages you've entered but not sent) that I was thinking would stay in the client. Because by the time you've built all the state into the server, you may as well handle the message transport too and forgo XMPP altogether, or have a gateway that allows XMPP through.

The TCP socket limitation is certainly an issue; we'd have to consider the limitations this would prevent in terms of deployment. Some advantages of BOSH are that it doesn't open a connection to the web server, and the XMPP connections aren't TCP connections kept open all the way to the browser. That makes scale much easier to manage; the downside being that there are connections from the browser going to multiple servers and the loss of state on page loads.

I'm starting to grok why this feature hasn't been addressed sooner.

@bradkoehn You misunderstand. The browser does not open TCP connections to remotes. TCP connections are managed by the XMPP server. It establishes connections to remote servers as needed. The component only maintains one stream: to the XMPP server.

I mentioned websockets as one way to push messages in realtime to the browsers. This means one websocket connection for every active user. Websockets are no requirement but they have less overhead than periodic HTTP requests that would be needed for polling (or BOSH for that matter, because then the client would have to speak proper XMPP, which is a silly requirement).

The state you are talking about is transient; there is no need to "build" any state "into the server". The client sends stuff to the component which then asks the XMPP server to handle the rest. I don't understand where you see complexity in this setup.

@jonnehass The only thing that may not scale too well is the need for a communication channel between browser and Diaspora server. There will be one websocket connection per browser. This is less overhead than periodically establishing a HTTP connection to poll for changes. Any production websocket server should be able to handle a high number of concurrent connections, though. This remains to be seen, but is not critical as the use of websockets is entirely optional in this scenario.

alternative to "just" websockets is something that falls back to older browsers, like socket.io

Nice Diaspora WebRTC chat feature https://diasp.de/posts/1384079

Interesting :)

@diaspeu I played with that, but I am not sure whether I should continue developing

cause of the Mozilla Loop project.

What is the opinion about that?

Should we use such alternative and think about only implementing the "address-book"?

( If webrtc is the thing we want ;-) )

Regards,
Lukas

After reading all comments here, basically we all agree that Chat is a very important feature and it must to be implemented.

Why we don't just make it as simple as possible in it first version?

As others my opinion is that Diaspora got a XMPP client from within that allow users to setup any XMPP account.

In diasporabr.com.br POD we already setup a Prosody XMPP server integrated with Diaspora database to use it's accounts and passwords.

I have to say that Prosody is a very nice piece of software.

If each POD install it's own XMPP server and Diaspora got a XMPP client the main issue - get a chat in Diaspora - is solved.

I agree that WebRTC is the most beautiful and featured option, but this will probably take quite a time to implement.

As you can see this topic has been discussed for more that two years now. Lets implement it as easy as possible.

I vote for a simple XMPP Client within Diaspora.

Hey,

This is my first time in Loomio so I'm curious about the vote feature.
How does it work? In some point the voting button will be enabled by some one? It is intelligent enough to read each message and figure it out?

Some can give me some insights?

Thanks

Hi Anahuac, and welcome to Loomio.

The voting function is only available when there is an open proposal. On this discussion there is only a closed proposal, which ended a year ago.

If you look on discussions with open proposals (you will see them at the top of your discussion list, and with coloured pie charts next to them) you will see how you can vote on those proposals.

Anyone can start a proposal, but please read our wiki article on discussion and voting first.

Hope that helps.

Hi Goob,

I see... thanks for your answer.

So what we are discussing here are not under voting, so what are we doing?

Is this to get some insights to the subject?

I mean, I'm not a Ruby developer, so I can't help in the Chat development, but how kind of effort I need to do to see Chat implemented on Diaspora?

I'm in charge of the diasporabr.com.br POD and every single day I got a couple of people asking me about chat and personal pages.

I really love the idea of offering both to users.

Regards

Well, the discussion about how to implement chat has taken place a long time ago (look at the previous pages on this thread). It now just needs people to volunteer to start working on it! Several people have offered, but always something has happened to stop them (other commitments, and so on).

Unfortunately, like you I'm not a coder so I can't help. If you know anyone who has those skills and might be willing to work on this, send them this way!

Well, I know that this is probably not the right place to ask, but... is there, anywhere, the API documentation?

Hi Anahuac, good to see you here, welcome!
As I see you've successfully implemented XMPP on your pod, so even if we didn't decided here which service to use for the chat yet (XMPP or WebRTC or whatever), how about telling us your first impressions using Prosody as the XMPP base? I mean, you could tell us about the use of server's resources, the capacity to handle many concurrent users, the integration between the XMPP server and the Diaspora handles, and so on.
I have the very same issue here: many users are always asking me about a chat implementation and I'm wondering if I also implement a XMPP server until this decision is made by the community, and until the final solution is ready for the Diaspora pods by the code.

Hi Vostok,

So, my very first impressions of have Prosody as XMPP server are grate. This is a very light program and the server resources have not been consumed at all. But I have to say that no many accounts are using it, so I really can't say how the server will deal with thousands of users.

The main reason to most of the account don't use XMPP is that regular users don't want to install clients out from diaspora and this is why I would love to see a XMPP client integrated on Diaspora.

Abut the technical integration it was a little bit tricky. A had to touch a lot the SQL plugin file to allow prosody authenticate against Diaspora password table.
I have to write a howto and publish it on thw diasporafoudation.org wiki and I will do it soon. Problably next week, because I'm going out for a little vacation for the Saint John holiday in Brazil.

Finally I got some time to do the Prosody + Diaspora howto. It is here:

https://wiki.diasporafoundation.org/Integration/XMPP/Prosody

take a look at it and give me some feedback

Hey Anahuac, thanks a lot for doing this. I hope you don't mind, I've just gone through and edited the English (not much to do, you speak it pretty well!) and added inline code markup where I thought this was appropriate.

Do check I've not added any errors or confusion: here's the diff.

@anahuac it looks really nice!

The next step is to automatically add a contact in XMPP when someone adds it to his aspects.

Then we will have to add a chat in the interface of diaspora*, maybe with something like jappix mini?

@goob I do not mind! In fact I appreciate it very much. Thank you for the contribution :-)

@flaburgan This is the idea! Now developers need to figure how to implement a XMPP client inside Diaspora interface :-)

Wow this is pretty awesome work @anahuac ! Will definitely give it a try on my pod.

Anyone want to work on integrating a chat client on the d* UI side? It should imho not be tied strongly to this Prosody solution though.

@jasonrobinson I'm glad you like it. This is how I can contribute to our beloved Diaspora :-)

As I said before, I'm very excited to see a chat client on the d* UI.

Oh @anahuac , thank you, thank you, thank you!

This is exactly what I was looking for.

I agree with @jasonrobinson that an integrated chat client should not be tied strongly to this Prosody solution though. However it wouldn't bother me at all since Prosody is my choice anyhow.

A XMPP chat dev for D* has started bitween Lukas @Zauberstuhl and me.

The solution use Vines ruby gem for XMPP server and Jappix Mini as frontend. Lukas is going to do an official pull request in the next couple of days.

@augier that sounds great!

Which solution would be the best ? Prosody, or Vines ?

@augier Vines looks really cool.

However, one thing that I don't understand is that some of the instructions imply the need for CouchDB, but it doesn't state that in all of the instructions. If anybody can confirm that we don't need CouchDB to make use of it for Diaspora, Vines would definitely get my vote.

Prosody would be awesome.

@anahuac thank you for publishing the howto on the wiki, nice work man!
I've implemented a XMPP server with Prosody on the pod and it's working pretty well so far.

@augier wonderful news!
The authentication between Diaspora SQL database and Prosody XMPP server is working fine by the auth script modified by Anahuac.
About Vines, I've seen it's similar to Prosody but it also allows the config of components (access to subdomains using passwords) and clustering of server instances (load-balancing using Redis) which I'm not sure if Prosody allows. Maybe Vines has more features than Prosody, need to be checked.

@seantilleycommunit I found this, not sure if helps:
"Vines can store user data in several databases out of the box: Apache CouchDB, Redis, MongoDB, any SQL database, or simply in the local file system."

@vostok Thanks! As I said before this is my small contribution, since I'm not a Ruby developer.

@augier wonderful news indeed!
My only concern is about performance. Did you, or anyone else, have some comparative between Vines and Prosody?

What makes me choose prosody was the fact that the jabber-br.org XMPP server migrated to it a couple of months ago exactly for a performance issue. The second reason was the fact that Prosody is developed in Lua that is a fast and very robust dev language.

@anahuac : Nope, no comparative. We weren't aware about the existance of Prosody. But as the interface is totally separated from the server, we can change with only few modifications. But the question is interesting. I only followed the work started by Lukas to add Jappix Mini.

@seantilleycommunit : Some features need CouchDB, indeed, as seen on the website. But I didn't deepen the question further.

@augier What about do it in a way that you can choose any XMPP server? I know this mean more work, but I think this will be grate stuff to have since each POD could choose the best server for it's own needs.

I'm very excited to see this up and running. Keep doing what you are doing and let us know of yours progress.

What about do it in a way that you can choose any XMPP server? I know this mean more work, but I think this will be grate stuff to have since each POD could choose the best server for it’s own needs.

It would be good if the XMPP server would be configurable by the podmin - so separating the chat UI/code and actual server parts. Reason - many podmins already run an XMPP server.

Awesome work guys - totally looking forward to this! You will be heroes if you finish this :)

@jason-robinson : The UI is totally separated from the server. So yes, it is possible.

@anahuac : even though this is perfectly possible, let's make it work with one server for the moment ^{^}
Then, when can write a standard JS interface to connect the UI and the server, and a ruby script to do the configuration, yes !

There's a lot of work in here ! :D

@augier it's perfectly ok by me :-) keep doing it!

Sure, small steps, then iterate :)

A first version is finished now. See demo here : https://www.youtube.com/watch?v=eLN0PK7qmXw

That is super awesome :) Do you guys have a pull request to submit so that review could start? Doesn't have to be in any way finished for merging.

I created a test account in sechat. But I am seeing "Error" in the chatbox. See attachment. Problem in my side?

@balasankarchelamat your browser is blocking the http-bind request cause it is currently non-ssl.

I am fixing that today! It should work with chrome and firefox (if you disable the security check).

Cheers

@lukasmatt Do you have a route by which people can provide bug reports? Here isn't really the right place, and I've started receiving bug reports to Diaspora HQ. If you've got a GitHub repo that would be perfect - alternatively is there a Diaspora account at which you're happy to receive them?

Feel free to report to:
https://github.com/zauberstuhl/diaspora

@jasonrobinson my next update includes some configuration changes which will give the podmin the ability to disable the chat or the internal server. That means as long the podmin knows what he is doing he can have his own xmpp server.. that is the awesome thing with XMPP it's a standard... no need for big changes :)

@lukasmatt : @goob is right : we should make a D* page for help and bug submitting. Currently, If you click on the Jappix logo, it redirects you to the Jappix Mini page. I'll try to make a D* help page tonight et modify the JS accordingly.

Thanks. That will help you, I'm sure, to get useful feedback on your good work.

Hey all.. what about this: https://github.com/diaspora/diaspora/pull/5073 ? All I have to do is follow this instructions?

@anahuac it's WIP still but sure, if you want to try that is awesome, to help the developers. But just don't expect it to be stable with d* :)

@anahuac It might be a good idea to reach out to the author of the pull request in case you have any questions. :)

@anahuac : Not sure the patch is still up to date since the code has changed.

Is it a joke ? diasporabrazil.org hasn't implemented anything. It has just installed the project currently and officially developped by diaspora*. There already is github repos for the development of an XMPP chat. See here for front-end and here for back-end. Chat has been added to next-major release. Just look at a few posts ago. I don't see the need for a new vote here...

That's what I also thought, Augier. We don't need another proposal but a qualified pull-request.

@rasmusfuhse : The pull request exists also thouh it is not mergable yet ;)

It will be used Heroku as XMPP server? It seems that there will be a centralized server with the possibility of make a new server from each node, but no as default. If it were thus, we will lose descentralization...
It's great that the project is in development, but maybe you can consider my worry.

It will be used Heroku as XMPP server? It seems that there will be a centralized server with the possibility of make a new server from each node, but no as default. If it were thus, we will lose descentralization…
It’s great that the project is in development, but maybe you can consider my worry.

As I understood, problem of Heroku is that port 6222 (usual XMPP port) is blocked. So pods hosted on Heroku won't be able to run an XMPP server.

All other can. Chat is already used on sechat.org, @lukasmatt's own pod and diasporabrazil.org. I think there is one or two others. When it's finished, all dispora pod wil have an embedded XMPP server in the form of a gem. Up to them to enable it or not.

So no, there's no centralization to worry.

OOh, thanks for your explanation and your contribution to this project, I was waiting for that function and wondering when it will be applied, now I know that the development is advanced and as I was thinkin, with XMPP as server and client on each node. Thank you again and sorry for make a new proposal, I wasn't understood...

No problem.
Take time to read the last few posts next time ;)

Yes, I'm new on loomio and don't saw that was more posts. It would be great also if you post the advances on #diaspora-hq to avoid these mistakes...

Well, I'm not sure if it's time yet for an official post. Though chat has been added to next major, its release is not certain. This is a big feature that needs to be tested before official release. It should be a big disappointment for user if it was eventually postponed.

ooh, there's no release date? well... at least, it's in development.

Yes, there is. Next-major should be coming at the beginning of 2015 if I'm not wrong. But next-major also contains vital improvements to the federation. So if chat isn't finished until then, it won't be possible to postpone next-release.

Good new, the chat is now merged in develop branch. But it needs to be polished, so that doesn't mean it will be launched in the next release.

Excellent :-)

A very good article which might be interesting to the participants of the discussion:

http://www.html5rocks.com/en/tutorials/webrtc/infrastructure/

i don't know if this is the right point for this, but anyway my question:
how big would be the effort of enabling webrtc in the chat (as it is already included in jsxc?