NiFi User Authentication with LDAP on Hortonworks sandbox

Prerequisite:

You must have Nifi or otherwise under hortonworks sandbox

Actions--> Add service-->nifi

Make sure Knox is installed on your sandbox and demo LDAP is started via Ambari

Steps:

First create Certification Authorities and client certificates under www.tinycert.org. Or use my certificate that I already created and download it under this url https://community.hortonworks.com/content/idea/73207/certification-authorities-and-client-certificates.html. Or follow this url https://community.hortonworks.com/content/kbentry/886/securing-nifi-step-by-step.html for creating certificates
Upload the cert-browser.pfx into your browser
Upload other two certificates to Sandbox with specific folder ex:”/root/certificate” and run these commands

# cd /root/scripts/

# mv cert.pfx cert.p12

# openssl x509 -outform der -in cacert.pem -out cacert.der

# keytool -import -keystore cacert.jks -file cacert.der

keystore is saved as ‘/root/certificate/cert.p12’ and a truststore is saved as ‘/root/certificate/cacert.jks’. and password is set as hadoop
Update Nifi configuration under ambari-->configs-->Advanced nifi-properties-env or run these command

# vi /opt/nifi-0.5.1.1.1.2.0-32/conf/nifi.properties

Under nifi.properties upadate https port and certificate details

# web properties #

nifi.web.war.directory=./lib

nifi.web.http.host=

nifi.web.http.port=

nifi.web.https.host=

nifi.web.https.port={{nifi_port}}

nifi.web.jetty.working.directory=./work/jetty

nifi.web.jetty.threads=200

# security properties #

nifi.sensitive.props.key=somekeyvalue

nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL

nifi.sensitive.props.provider=BC

nifi.security.keystore=/root/certificate/cert.p12

nifi.security.keystoreType=PKCS12

nifi.security.keystorePasswd=hadoop // keystore and key pwd must be common

nifi.security.keyPasswd=hadoop

nifi.security.truststore=/root/certificate/cacert.jks

nifi.security.truststoreType=JKS

nifi.security.truststorePasswd=hadoop

nifi.login.identity.provider.configuration.file=/opt/nifi-0.5.1.1.1.2.0-32/conf/login-identity-providers.xml

nifi.security.user.login.identity.provider=ldap-provider

nifi.security.needClientAuth=true

nifi.security.user.credential.cache.duration=24 hours

nifi.security.user.authority.provider=file-provider

nifi.security.support.new.account.requests=true

Now configure the authorized users in ‘authorized-users.xml’ file this is to login as NiFi Administrator, every other users can be pulled from LDAP after this administrator assigns roles on request, configuration of user is based on certificate.

# vi /opt/nifi-0.5.1.1.1.2.0-32/conf/authorized-users.xml

For example:

<role name="ROLE_ADMIN"/>
</user>

Now configure ./conf/login-identity-providers.xml with reference to Knox Demo LDAP Server

<provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.ldap.LdapProvider</class>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">uid=admin,ou=people,dc=hadoop,dc=apache,dc=org</property>
<property name="Manager Password">admin-password</property>
<property name="TLS - Keystore">/root/scripts/cert.p12</property>
<property name="TLS - Keystore Password">hadoop</property>
<property name="TLS - Keystore Type">PKCS12</property>
<property name="TLS - Truststore">/root/scripts/cacert.jks</property>
<property name="TLS - Truststore Password">hadoop</property>
<property name="TLS - Truststore Type">JKS</property>
<property name="TLS - Client Auth"></property>
<property name="TLS - Protocol">TLS</property>
<property name="TLS - Shutdown Gracefully"></property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url">ldap://localhost:33389</property>
<property name="User Search Base">ou=people,dc=hadoop,dc=apache,dc=org</property>
<property name="User Search Filter">uid={0}</property>
<property name="Authentication Expiration">12 hours</property>
</provider>

Once configuration is over now restart Nifi

# /opt//nifi-0.5.1.1.1.2.0-32/bin/nifi.sh restart

Now open ‘Chrome’ browser and load client certificate associated with ADMIN user and login to secure https url of NiFi running on sandbox:

https://localhost:9090.

Open another browser to establish another session

https://localhost:9090.

It will popup below screen for login, enter credentials for accounts part of LDAP. Below are credentials part of knox demo ldap we have configured.

tom/tom-password

admin/admin-password

sam/sam-password

guest/guest-password

Enter the password and hit login, it will show that request is pending for approval
Now navigate to the non-secure (http) address for your NiFi which lets everyone in as anonymous with full access. Go to the user management UI via the icon and grant your user the "admin" role.
Configure your nifi.properties file for secure (https) and non-secure (http) access. You will need to use unique ports for each. (9090 for http and 9091 for https for example).

Make configure in nifi.properties for both secure (https) and non-secure (http)

Secure (https)--> http://pastie.org/10984633.
Non-secure (http)--> http://pastie.org/10984635.

Now as ‘Demo‘ user in Nifi Administrator assign role to Tom(LDAP users)
Now go back to the old session as tom , refresh the browser and it will be logged in as tom with privileges assigned by NiFi administrator. Test it for other users as well.

REST API WITH POSTGRES DATABASE

First i started working on eclipse mars ,but i got lot of errors even at sample program that given in https://www.tutorialspoint.com/restful/restful_first_application.htm. Main drop back of eclipse is we need to add many jersey files and jdk 1.7 , tomcat into application
Then i move on to netbeans because i contain in bulid jersey files, jdk , tomcat.

Netbeans setup Environment

https://netbeans.org/downloads/start.html?platform=linux&lang=en&option=all

Give Executable permission : $ chmod +x ~/Downloads/netbeans-8.1*.sh

Start Installer : $ cd ~/Downloads && ./netbeans-8.1*.sh

Set JDK location : Browse to JDK location

-----------------------------------------------------------------------

Tomcat server setup Environment

http://tomcat.apache.org/download-80.cgi => Find Full Documentation => .tar.gz => Download

$ tar –zxvf apache-tomcat-8.5.5.tar.gz

$ cd apache-tomcat-8.5.5/bin/

To start tomcat server:

$ ./startup.sh

Browse Tomcat running or not:

Localhost:8080

To stop tomcat server:

$ ./shutdown.sh

To change tomcat users name and password :

$ gedit /path/to/apache-tomcat-8.5.5/conf/tomcat-users.xml

To change server port number:

$ gedit /path/to/apache-tomcat-8.5.5/conf/server.xml

------------------------------------------------------------------------------------------------------------------------

Create Rest Api application

File=>New Project=>Java Web=>Web Application=>Next => Enter Project Name => next=>Select Server (Either GlassFish/Tomcat) => next=>Finish.

http://ayazroomy-java.blogspot.in/2013/08/java-webservice-tutorial-part-11.html

https://www.tutorialspoint.com/restful/restful_first_application.htm

https://dzone.com/articles/restful-web-services-java

http://javapapers.com/java/restful-web-services-with-java-jax-rs-using-jersey/

-----------------------------------------------------------------------

Postgres setup Environment

1. sudo apt-get update
2. sudo apt-get install postgresql postgresql-contrib

3. Create a user account called postgres which is associated to postgres role

sudo -i -u postgres

4. Postgre is prompt by immediately typing

psql

5. For Exit Postgres type

6. For enter into Postgres use this keyword

sudo -u postgres psql postgres

7. For creating database password

\password postgres

8. Create database in Postgres

create database databasename

9. For connecting database

\c databasename

10. For disconnecting database

----------------------------------------------------------------------------------------------------

SETTING PASSWORD FOR POSTGRES:

supranimbus04@supranimbus04-Inspiron-3250:~$ sudo -u postgres psql

Password:

It will ask password , If the configuration of file $ sudo gedit /etc/postgresql/9.3/main/pg_hba.conf

local all postgres md5

If we don’t need to password , then change configuration of the

$ sudo gedit/etc/postgresql/9.3/main/pg_hba.conf

local all postgres peer

If any changes made into $ sudo gedit/etc/postgresql/9.3/main/pg_hba.conf file , Then you need to restart to the service

sudo service postgresql restart

--------------------------------------------------------------------------------------------------------

To view postgres status

$ sudo service postgresql status

--------------------------------------------------------------------------------------------------------

Creating table in postgres

Create table tablename ();

postgres=# create table testtbl(id int primary key not null,name char(30));

2. To view list of tables

\d - is to view list of new tables that are created
\dt - is to view list of tables that are present in database

3. To delete table

Drop table tablename;

------------------------------------------------------------------------------------------------------

For JDBC connection

https://jdbc.postgresql.org/download.html visit this link to download “JDBC3 Postgresql Driver, Version 9.3-1103” driver
Now add this driver into netbeans libraries
For reference visit this link

http://java2db.com/java-integrations/jdbc-with-postgresql-using-myeclipse-and-netbeans

--------------------------------------------------------------------------------------------------------

Employee details program using Rest api with postgres database

Access.java

DataBase.java

Employee.java

AccessManager.java

EmployeeSevice.java

Output

JSON file will be displayed under this URL

http://localhost:8282/Employees/webresources/employeeService/employees

Narasimma Varman

Wednesday, 21 December 2016