Data Persistence & Security

 

ORM and Security

The figure above shows the growing layers of our application. By adding MyBatis, we now have a translation from the Java and Spring world to the world of databases and SQL.

Why a database is required?

Adding a database to our application is a way to externalize data persistence problems. When storing data in memory at runtime, we struggle to deal with:

• Storage Space
• Concurrency
• Persistence of Data

Using a database allows us to isolate these concerns from the rest of our application, so we can focus on the business logic of our application.

How to interact with a database?

There are many ways to manage the communication between an application and a database. For this time, we’ll be using the library MyBatis to transform Java objects to SQL query parameters and to transform SQL query results into Java objects.

We’ll create MyBatis Mappers as Spring beans, so we can inject them into any other beans that need them. For example, if we think about an online-shopping example, we might have a UserMapper that manages credentials and profile information and a CartMapper that manages the contents of an individual user’s cart. We can inject the UserMapper into a Checkout Service that also receives the CartMapper to apply the charges in a User’s cart to that User’s stored payment information.

Later, we’ll combine our MyBatis mappers with Spring Security to authenticate each user’s session automatically. To continue our earlier example, this means we could inject the UserMapper in some kind of Authentication Service to check client credentials on login.

Additional Reading

• Briefly read the research paper titled Object/relational mapping 2008: hibernate and the entity data model by Elizabeth O’Neil for a better insight into the ORM.
• If you are not familiar with JDBC, refer to this official JDBC Introduction, and have a quick look at the underlying architecture. Though, we will not use the JDBC directly. MyBatis will automatically generate JDBC requests.

ORM and Security

What Data Should be Stored in a Database?

• Data shared across multiple user sessions, like a product inventory
• Persistent data that should remain accessible after being logged out, like user profile or shopping cart

How Should Data be Structured?

• Intuitively. Most data can be stored in a similar format to the data objects that represent it in Java, with attributes matching column names.
• Differing. Some data must be stored differently for security reasons, such as encrypted passwords. Other data may require a different format for efficient storage, such as large files.

Thinking about Security

The main question to ask is: “What pages can a user access?”

• User-Specific Data
• Generally Accessible (Unsecured) Data
• May Vary by Domain

ORM is the Norm

The first step in using ORM is to define the data model. Consider the relationship between the following tables:

• user - A table containing the user ID, username, and password for each user.
• user_cart - We are assuming, any user can have one cart at max. The cart would have multiple items (inventory). This table stores the user ID, inventory ID, and the quantity of inventory the user wants to purchase. In this table, the user ID, and inventory ID are the foreign keys respectively. Together, the combination of the user ID and inventory ID can act as a primary key.
• inventory - It stores the inventory ID, name, and unit price of each inventory.

We can represent their relationship in SQL with this image below.

Relationship Between SQL Tables — Image showing the relationship between the user, user_cart, and inventory tables in SQL

A primary feature of ORM is that this type of relationship should have a natural mapping to Java classes. We can represent this same data in Java using a simple class diagram.

Class Diagram Corresponding to SQL Tables — The class diagram shows that the columns of the SQL diagram becomes fields of the Java objects User, CartItem, and InventoryItem

The data types of these class attributes correspond to the data types of the SQL columns. Some Java types can be mapped to many different SQL types, and some SQL types can be mapped to multiple Java types, but in this case the type mappings are obvious. For a full list of the MyBatis type mapping, consult the MyBatis 3 TypeHandlers list.

Once you have defined your data types, MyBatis can automatically map Java objects to SQL statements.

The diagram above shows the ORM Process Visualization, displaying how createUser(user) is converted into SQL that inserts a row in a table, and getUserByName(name) is converted into SQL that returns a result which becomes a Java object.

Key Terms

• ORM: Object-Relational Mapping. A general term describing a set of technology that can be used to automatically convert data between database representation and application representation.
• Mapping: Drawing a relationship between a field in a Java class and a column in a SQL table.
• One to One: A relationship between two objects in which one entity is on each side of the relationship.
• Many to Many: A relationship between two objects in which multiple copies of each entity can be related to multiple copies of the other entity.

A primary key’s main features are:

• It must contain a unique value for each row of data.
• It cannot contain null values.
• Every row must have a primary key value.

Suggested Reading

• The book titled “Patterns of Enterprise Application Architecture” by Martin Fowler
• The Object-Relational Impedance Mismatch

Solution: ORM is the Norm

Erratum — At timestamp 1:02 in the video above, when the instructor says “Big Double”, he intends to say “BigDecimal”.

You should have created four classes for this exercise: Customer, Order, TacoOrder, and Delivery. These class files should live in a new data folder in the course1 folder. The variables in each class should correspond to the variables in the database tables. Here are some sample implementations:

public class Customer {
   private Integer id;
   private String userName;
   private String password;

   /* getters and setters not shown */
}

Pro Tip — It would be better if the password field for the Customer class is char[ ] array instead of a String. As a developer, you would want to encrypt each character so that you wouldn't store the actual password anywhere in the system. A String is immutable by nature, and hence you cannot change/convert the characters of a String.

public class Order {
   private Integer id;
   private Integer customerId;

   /* getters and setters not shown */
}

public class TacoOrder {
   private Integer orderId;
   private String tacoName;
   // this will work here, but you should often use BigDouble for prices 
   // if you plan to do any math with them
   private Double tacoPrice;
   private Integer count;

  /* getters and setters not shown */
}

Pro Tip — The java.math.BigDecimal data type is better than any of the primitive data types, including Double, for precise values, such as currency or tacoPrice.

public class Delivery {
   private Integer id;
   private Integer orderId;
  // there are a few types you can use for this. 
  // java.sql.Timestamp contains both date and time
   private Timestamp time;

   /* getters and setters not shown */
}

MyBatis Mappers

https://github.com/udacity/nd035-c1-spring-boot-basics-examples/tree/master/udacity-jwdnd-c1-snippets-master/src/main/java/com/udacity/jwdnd/c1/snippets/l4

MyBatis overview

MyBatis provides a shallow ORM layer over JDBC (Java Database Connectivity). That means it helps map your Java objects to queries that save and retrieve data using JDBC.

MyBatis is mostly used through interface definitions. MyBatis automatically generates classes that implement the interface and makes them available as Spring beans. This is an example interface that defines a MyBatis Mapper.

@Mapper
public interface UserMapper {
   @Select("SELECT * FROM USERS WHERE username = #{username}")
   User getUser(String username);
}

This code above uses #{username} to identify the username parameter. It's like Thymeleaf parameters, but for SQL!

For more information on the template syntax MyBatis uses for SQL, check out the official documentation. There are additional annotations for @Insert, @Update, and @Delete as well.

MyBatis Demo

@Insert and @Options annotation

The @Insert annotation automatically references attributes on the user object. Note username, firstName, lastName are all referenced directly here, because they are attributes of the user object.

@Mapper
public interface UserMapper {
   @Insert("INSERT INTO USERS (username, salt, password, firstname, lastname) " +
           "VALUES(#{username}, #{salt}, #{password}, #{firstName}, #{lastName})")
   @Options(useGeneratedKeys = true, keyProperty = "userId")
   int insert(User user);
}

This example also demonstrates the @Options annotation. @Insert normally returns an integer that is the count of rows affected. By using the @Options annotation, we're telling MyBatis that we want to automatically generate a new key and put it in userId. Now the method will return the new userId once the row has been inserted.

All we have to do to use these methods is inject beans for this interface into our services and MyBatis will automatically create the code for the JDBC requests!

The diagram above shows that MyBatis Mappers lie at the center of our onion architecture. Remember, that means that the only beans that should have dependencies on them are in the next layer up, services.

Key Terms

• @Select, @Insert, @Update, @Delete: Annotations representing SQL statements to be executed. Each annotation takes a string for a SQL statement of the corresponding type. For example, a @Select annotation takes a string for a SQL SELECT statement.
• @Options: Annotation providing access to switches and configuration options for JDBC statements.

Further Research

• For a full list of the available MyBatis annotations and some example usage, see the MyBatis Java API documentation.
• For an informal overview of result mapping with MyBatis annotations, see this Medium article

Solution: MyBatis Mappers

Explanation

We declared three new methods in our interface. In this solution, they’re named findDelivery(), insert(), and delete(), but you can name them as you like.

@Mapper
public interface DeliveryMapper {
   @Select("SELECT * FROM Delivery WHERE id = #{id}")
   Delivery findDelivery(Integer id);

   @Insert("INSERT INTO Delivery (orderId, time) VALUES(#{orderId}, #{time})")
   @Options(useGeneratedKeys = true, keyProperty = "id")
   Integer insert(Delivery delivery);

   @Delete("DELETE FROM Delivery WHERE id = #{id}")
   void delete(Integer id);
}

The code shown above has the following annotations:

1. The first annotation is a simple @Select like we saw earlier in the lesson.
2. The second annotation @Insert is pretty similar to the earlier example as well. It usesINSERTcommand to create a new row in the Delivery table. Note that it only needs to provide the order ID and time values, because the ID itself is generated.
3. The @Options annotation indicates to generate the key for the id property and return it from the method. Also, note that our VALUES portion of the query just provides order ID and time directly. MyBatis can figure out that they are attributes of our Delivery object.
4. Lastly, @Delete is very similar to the @Select, but make sure you use the right annotation for it!

Practical Example: User Credentials and Authentication

Authentication in a web application

User support is a common feature in web applications, which means that a user can register an account and use credentials to login to the application in the future.

It’s important to design databases with the assumption that they will someday be breached, and so we cannot store passwords or other secret credentials in plain text. Two approaches to storing passwords are:

• Encryption: Modifying data before storing it, with the intention of using another algorithm to return the data to its original form once it needs to be used.
• Hashing: Modifying data before storing it with the intention of never returning it to its original form. The modified data will be compared to other modified data only.

Hashing and Encryption should occur in a service dedicated to that purpose, rather than on the front end or in the controller. Hashing sometimes makes use of another technique, Salting.

The flow of data

Remember the separation of concerns and our onion architecture! The idea is that all user flows originate externally, travel through a controller, then through one or more services, finally through a data access bean to the database, and then all the way back up the chain. Structuring applications this way makes it easy to follow dependencies and separate concerns, so that’s how we’re going to build applications from now on.

Authentication and Related Services — Demo I

For the full lecture sample code from this video and the next, check here

Hashing Implementation

Below is an example of how to hash user passwords in the database. First, we have the User class and the UserMapper. When our UserService creates a new user, it uses a hashing service to convert the password to a hashed value before saving it.

public class User {
    private Integer userId;
    private String username;
    private String salt;
    private String password;
    private String firstName;
    private String lastName;
    /* constructor, getters, and setters omitted */
}

@Mapper
public interface UserMapper {
    @Select("SELECT * FROM USERS WHERE username = #{username}")
    User getUser(String username);

    @Insert("INSERT INTO USERS (username, salt, password, firstname, lastname) VALUES(#{username}, #{salt}, #{password}, #{firstName}, #{lastName})")
    @Options(useGeneratedKeys = true, keyProperty = "userId")
    int insert(User user);
}

public int createUser(User user) {
    SecureRandom random = new SecureRandom();
    byte[] salt = new byte[16];
    random.nextBytes(salt);
    String encodedSalt = Base64.getEncoder().encodeToString(salt);
    String hashedPassword = hashService.getHashedValue(user.getPassword(), encodedSalt);
    return userMapper.insert(new User(null, user.getUsername(), encodedSalt, hashedPassword, user.getFirstName(), user.getLastName()));
}

The hashing service itself has a single method that takes some data and salt and creates a string representing the hashed value.

• Salt: random data that is combined with the input string when hashing so that the resultant hashed values are unique for each row. This means that two users with the same password would not have the same hash in the database.

public String getHashedValue(String data, String salt) {
    byte[] hashedValue = null;

    KeySpec spec = new PBEKeySpec(data.toCharArray(), salt.getBytes(), 5000, 128);
    try {
        SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
        hashedValue = factory.generateSecret(spec).getEncoded();
    } catch (InvalidKeySpecException | NoSuchAlgorithmException e) {
        logger.error(e.getMessage());
    }

    return Base64.getEncoder().encodeToString(hashedValue);
}

Authentication Implementation

When a user logs in, we have no way to retrieve their original password, but we can re-hash their user input and see if it matches the hashed value in our database. Below is an example AuthenticationService class that implements a Spring interface, the AuthenticationProvider. This allows Spring to integrate our provider with many different authentication schemes, but we can see in our supports method that we specify that we only support UsernamePasswordAuthentication.

The authenticate() method takes an Authentication object from spring and returns an authentication token if the user's credentials are correct.

@Service
public class AuthenticationService implements AuthenticationProvider {
    private UserMapper userMapper;
    private HashService hashService;

    public AuthenticationService(UserMapper userMapper, HashService hashService) {
        this.userMapper = userMapper;
        this.hashService = hashService;
    }

    @Override
    public Authentication authenticate(Authentication authentication) throws AuthenticationException {
        String username = authentication.getName();
        String password = authentication.getCredentials().toString();

        User user = userMapper.getUser(username);
        if (user != null) {
            String encodedSalt = user.getSalt();
            String hashedPassword = hashService.getHashedValue(password, encodedSalt);
            if (user.getPassword().equals(hashedPassword)) {
                return new UsernamePasswordAuthenticationToken(username, password, new ArrayList<>());
            }
        }

        return null;
    }

    @Override
    public boolean supports(Class<?> authentication) {
        return authentication.equals(UsernamePasswordAuthenticationToken.class);
    }
}

Authentication and Related Services — Demo II

https://github.com/udacity/nd035-c1-spring-boot-basics-examples/tree/master/udacity-jwdnd-c1-l4-spring-security-basics-master

In order for Spring to actually use our AuthenticationService, we need to extend our Web Security configuration. We do that with an adapter for the WebSecurityConfigurer. This example overrides two configure methods:

• configure(AuthenticationManagerBuilder auth): used to tell Spring to use our AuthenticationService to check user logins
• configure(HttpSecurity http): used to configure the HttpSecurity object by chaining methods to express security requirements

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {

    private AuthenticationService authenticationService;

    public SecurityConfig(AuthenticationService authenticationService) {
        this.authenticationService = authenticationService;
    }

    @Override
    protected void configure(AuthenticationManagerBuilder auth) {
        auth.authenticationProvider(this.authenticationService);
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
                .antMatchers("/signup", "/css/**", "/js/**").permitAll()
                .anyRequest().authenticated();

        http.formLogin()
                .loginPage("/login")
                .permitAll();

        http.formLogin()
                .defaultSuccessUrl("/home", true);
    }
}

We can see that the second configure method does four things:

• Allows all users to access the /signup page, as well as the css and js files.
• Allows authenticated users to make any request that’s not explicitly covered elsewhere.
• Generates a login form at /login and allows anyone to access it.
• Redirects successful logins to the /home page.

https://github.com/resisttheurge/udacity-jwdnd-c1-l4-spring-security-basics/blob/master/src/main/java/com/udacity/jwdnd/spring_security_basics/service/HashService.java

Key Terms

• Onion Pattern: Sometimes also called Tiered Architecture, Multi-tiered Architecture, or n-tiered Architecture. This is a design pattern that separates areas of the application into controller, service, and data layers (and sometimes more). User flows originate from the controller tier, which passes them to the service tier, which then reaches a data access bean.
• Encryption: Modifying data before storing it, with the intention of using another algorithm to return the data to its original form once it needs to be used.
• Hashing: Modifying data before storing with the intention of never returning it to its original form. The modified data will be compared to other modified data only.
• Salt: random data that is combined with the input string when hashing so that the resultant hashed values are unique for each row. This means that two users with the same password would not have the same hash in the database.

Further Research

• Spring Security Java Configuration
• HttpSecurity JavaDoc
• learn more about encryption, hashing, and salts, check out this high-level explanation.

Solution: User Credentials and Authentication

Here is an example configure method that meets our requirements:

@Override
protected void configure(HttpSecurity http) throws Exception {
   http.authorizeRequests()
           .antMatchers("/order", "/css/**", "/js/**").permitAll()
           .anyRequest().authenticated();
   http.formLogin()
           .loginPage("/login")
           .permitAll();
   http.formLogin()
           .defaultSuccessUrl("/tacos", true);
}

The first method chain permits all requests to the /order page or to our css and js directories, and then it allows authenticated users to make any kind of request. The next chain permits all users to access the auto-generated login page at /login. Remember that Spring creates this for us. Lastly, the third method chain redirects successful logins to the /tacos page by default.

Exercise: Final Review

For this final review, you’ll be updating your chat app from the last lesson to support user registration and login, as well as to store chat messages in a database rather than the in-memory list we created in the last lesson. That means you’re going to have done a couple of things:

1. Re-enable Spring Security. In the Last lesson, we commented out the Spring Security dependencies in the project pom.xml file. This time we need to add them, and MyBatis to the pom.xml back! As a reminder, these are the two dependencies you need to un-comment (or add back to your project if you deleted them last time):

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.security</groupId>
   <artifactId>spring-security-test</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.mybatis.spring.boot</groupId>
   <artifactId>mybatis-spring-boot-starter</artifactId>
   <version>2.1.2</version>
</dependency>

2. Define the database tables for the app.

CREATE TABLE IF NOT EXISTS USERS (
userid INT PRIMARY KEY auto_increment,
username VARCHAR(20),
salt VARCHAR,
password VARCHAR,
firstname VARCHAR(20),
lastname VARCHAR(20)
);
CREATE TABLE IF NOT EXISTS MESSAGES (
messageid INT PRIMARY KEY auto_increment,
username VARCHAR NOT NULL,
messagetext VARCHAR NOT NULL
);

1. We haven’t talked about this file much, but it’s been in all of the examples we’ve looked at this lesson. Spring Boot will, by default, look for this file when the server starts and execute it if it’s present. You can use this behavior for a variety of useful database-related tasks, but all we’re using it for here is to create two tables in the database — USERS and MESSAGES.
2. Create Java classes that match the database tables. If we want to take advantage of MyBatis, we need to create a User class that matches the data in the USERS table above, and we need to update the ChatMessage class from last lesson's final review to match the MESSAGES table above.
3. Create MyBatis mapper classes to handle interacting with the database. As we discussed this lesson, we need to create specialized @Mapper-annotated interfaces to access our database tables. You'll need two for this project - one for the USERS table and one for the MESSAGES table.
4. Add user registration and login support to the application, and require a login to access the /chat URL. Here's the link to the full project for reference. You'll have to implement the following elements from that example in this project as well:

• The AuthenticationService, HashService, and UserService classes
• The LoginController and SignupController classes
• The SecurityConfig class
• The login.html and signup.html HTML templates
• And (optionally) the /css and /js folders found in src/main/resources/static.

1. Most of these can be copied almost verbatim, but it’s good practice for you to check and make sure you understand what needs to be copied, what needs to be changed, and why.
2. Update the MessageService to use the database instead of an in-memory list of messages.That means it's going to have to use the MessageMapper you created!
3. Update the chat.html template to remove the username input from the message submission form. If we want the user to log in before seeing the chat page, we shouldn't make them enter their own username. Plus, we don't want to have users impersonating each other!
4. Update the ChatController to get the currently logged-in user's username upon message submission. You'll have to do some research for this one! It's common to need the currently-logged-in user's credentials when processing requests, so the solution shouldn't be too hard to find. You also need to attach the username to the message before passing it off to the UserService - but again, that's up to you to figure out.
5. [BONUS] add a logout button to the chat template. It may not seem incredibly important right now, but generally it’s actually very necessary to allow users to log out from an application. Many people across the world browse the internet from public computers in libraries and internet cafes, and even more share the same computer with friends and family. Our security implementation won’t be worth much if the next person who uses the app continues to use the previous person’s account! For this bonus task, you’ll again have to do some research, this time into how Spring Security handles logout actions.

Solution: Final Review

For the full solution code, check here.

Glossary

• ORM: Object-Relational Mapping. A general term describing a set of technology that can be used to automatically convert data between database representation and application representation.
• JDBC: Java Database Connectivity API, which is a specification for making SQL requests from Java.
• MyBatis: A thin ORM over JDBC that automatically generates code to execute SQL statements over JDBC and maps the results to Java objects.
• Mapping: Drawing a relationship between a field in a Java class and a column in a SQL table.
• One to One: A relationship between two objects in which one entity is on each side of the relationship.
• Many to Many: A relationship between two objects in which multiple copies of each entity can be related to multiple copies of the other entity.
• @Select, @Insert, @Update, @Delete: Annotations representing SQL statements to be executed. Each annotation takes a string for a SQL statement of the corresponding type. For example, a @Select annotation takes a string for a SQL SELECT statement.
• @Options: Annotation providing access to switches and configuration options for JDBC statements.
• Onion Pattern: Sometimes also called Tiered Architecture, Multi-tiered Architecture, or n-tiered Architecture. This is a design pattern that separates areas of the application into controller, service, and data layers (and sometimes more). User flows originate from the controller tier, which passes them to the service tier, which then reaches a data access bean.
• Encryption: Modifying data before storing it, with the intention of using another algorithm to return the data to its original form once it needs to be used.
• Hashing: Modifying data before storing it with the intention of never returning it to its original form. The modified data will be compared to other modified data only.
• Salt: random data that is combined with the input string when hashing so that the resultant hashed values are unique for each row. This means that two users with the same password would not have the same hash in the database.

Conclusion

• How to Identify Persistent Data
• Where to Store Persistent Data (SQL, in This Case)
• How to Use MyBatis, a Simple ORM Framework to Access Our Database
• How to Integrate MyBatis with Spring Through Simple Annotations

We also took a peek at Security!

• Explored a Real-World Example of Database Access
• Configured Simple Username/Password Authentication
• Connected Spring Security’s Configuration with Our User Credentials