There are 3 ways the client communicates with the server:

• The flash-client uses RTMP protocol to transfer Audio/Video and to send and receive the user data (login et cetera) to the server and back
• The browser uses HTTP protocol to load the SWF and to upload and download the files (documents, pdfs, images) to the server and back.
• The screensharing client uses RTMP protocol to transfer screen data and remote control to the server and back

Prerequisites

• You need OpenMeetings 1.9.x or later for this, OpenMeetings 1.8.x does not have those options.
• Install OpenMeetings according to the install instructions and check that it runs without problems
• Rename the existing keystore file red5/conf/keystore.jmx to red5/conf/keystore.bak
• Rename the existing truststore file red5/conf/truststore.jmx to red5/conf/truststore.bak

Create Keystore from the scratch (*)

1. Create a new keystore and key, use the same password for both:
   
   

   keytool -keysize 2048 -genkey -alias red5 -keyalg RSA -keystore red5/conf/keystore
   Enter keystore password:
   Re-enter new password:
   What is your first and last name?
   [Unknown]:  <your hostname, e.g demo.openmeetings.de>
   What is the name of your organizational unit?
   [Unknown]:  Dev
   What is the name of your organization?
   [Unknown]:  OpenMeetings
   What is the name of your City or Locality?
   [Unknown]:  Henderson
   What is the name of your State or Province?
   [Unknown]:  Nevada
   What is the two-letter country code for this unit?
   [Unknown]:  US
   Is CN=demo.openmeetings.de, OU=Dev, O=OpenMeetings, L=Henderson, ST=Nevada, C=US correct?
   [no]:  yes
   Enter key password for <red5>
   

2. Generate a CSR:

   keytool -certreq -keyalg RSA -alias red5 -file red5.csr -keystore red5/conf/keystore

3. Submit CSR to your CA of choice and receive a signed certificate
4. Import your chosen CA's root certificate into the keystore (may need to download it from their site - make sure to get the root CA and not the intermediate one):

   keytool -import -alias root -keystore red5/conf/keystore -trustcacerts -file root.crt

   (note: you may receive a warning that the certificate already exists in the system wide keystore - import anyway)
5. Import the intermediate certificate(s) you normally receive with the certificate:

   keytool -import -alias intermed -keystore red5/conf/keystore -trustcacerts -file intermediate.crt

6. Import the certificate you received:

   keytool -import -alias red5 -keystore red5/conf/keystore -trustcacerts -file demo.openmeetings.de.crt

7. Please NOTE according to this http://javarevisited.blogspot.com/2012/09/difference-between-truststore-vs-keyStore-Java-SSL.html guide you can split keystore and truststore (OPTIONAL you might just copy keystore to truststore)
8. Create additional certificate as described above. Add this certificate to the following keystores: red5/conf/keystore.screen and red5/conf/keystore. (This step is required to be able to use screen-sharing web application, you can copy "main" keystore while testing)

Create Keystore using existing key-pair (**)

Prerequicites:

• Server key: red5.key
• Signed CSR: red5.crt
• CA's root certificate: root.crt
• ** Intermediate certificate(s): intermedXX.crt

1. Export existing keys into PKCS12 format:
   
   

   openssl pkcs12 -export -in red5.crt -inkey red5.key -out red5.p12 -name red5 -certfile root.crt -certfile intermedXX.crt
   
   Enter Export Password: password
   Verifying - Enter Export Password: password
   

2. Import resulting red5.p12 into keystore:

   keytool -importkeystore -srcstorepass password -srckeystore red5.p12 -srcstoretype PKCS12 -deststorepass password -destkeystore red5/conf/keystore -alias red5
   

3. Import your chosen CA's root certificate into the keystore (may need to download it from their site - make sure to get the root CA and not the intermediate one):

   keytool -import -alias root -keystore red5/conf/keystore -trustcacerts -file root.crt

   (note: you may receive a warning that the certificate already exists in the system wide keystore - import anyway)
4. Import the intermediate certificate(s) you normally receive with the certificate:

   keytool -import -alias intermed -keystore red5/conf/keystore -trustcacerts -file intermedXX.crt

5. Please NOTE according to this http://javarevisited.blogspot.com/2012/09/difference-between-truststore-vs-keyStore-Java-SSL.html guide you can split keystore and truststore (OPTIONAL you might just copy keystore to truststore)
6. Create additional certificate as described above. Add this certificate to the following keystores: red5/conf/keystore.screen and red5/conf/keystore. (This step is required to be able to use screen-sharing web application, you can copy "main" keystore while testing)

SSL for the web interface

Please perform following steps if you want to use SSL for the web interface. This is mainly to secure the server against MITM attacks, additionally some other features like file uploads also use a plain HTTP connection if this is not done. The following instructions assume that you have already set up RTMPS successfully.

1. Edit red5/conf/jee-container.xml file:
   Comment Tomcat without SSL enabled section
   UNComment Tomcat with SSL enabled section
2. Edit red5/webapps/openmeetings/public/config.xml and set
   

   	<protocol>https</protocol>
   	<red5httpport>5443</red5httpport>
   

3. Restart red5 and try to connect to https://your.server:5443 - you should be redirected to the OpenMeetings app and all access should be via HTTPS and/or RTMPS (close port 5080 to be sure).

Set up RTMPS

Credits

Credits goto: Nexus and Holger Rabbach for their help and contribution and configuration documention!