Build your own AI Virtual Assistant With MERN STACK 🔥 | #4 Major Project

The video introduces an advanced virtual assistant built with the MERN stack. Unlike previous versions that required button clicks, this assistant activates by simply calling its name and provides intelligent, contextual responses using the Gemini API. Users can customize the assistant's name and image. The video demonstrates the registration, login, and customization process, including image selection and name assignment (e.g., 'Jarvis'). The assistant is shown responding to queries about its identity, date, time, and performing tasks like YouTube searches.

The tutorial begins with setting up the backend using VS Code. This involves creating a new project folder, '4. Virtual Assistant,' with separate 'backend' and 'frontend' subfolders. The backend is initialized with npm and essential packages like Express (for server and routing), Mongoose (for MongoDB integration), dotenv (for environment variables), nodemon (for automatic server restarts), jsonwebtoken (for authentication), bcrypt.js (for password hashing), cookie-parser (for cookie handling), Cloudinary (for image storage), and Multer (for file uploads) are installed.

The MongoDB Atlas database is set up. The process includes creating a new project, setting up a cluster, and configuring user credentials and network access to allow connections from anywhere. The MongoDB connection URL is then stored as an environment variable in a '.env' file. A dedicated 'db.js' file within the 'config' folder is created to handle the asynchronous database connection using Mongoose, ensuring a clean and modular setup.

A 'User' model is created in 'user.model.js' to define the schema for user data, including name, email, password, assistant name, assistant image, and a history array for interactions. Authentication controllers ('auth.controller.js') are then developed for user signup, login, and logout. These controllers utilize bcrypt.js for secure password hashing and jsonwebtoken for generating and managing user tokens, which are stored in HTTP-only cookies for session management.

API routes are defined in 'user.routes.js' for authentication (signup, login, logout) and for fetching the current user's data. Express Router is used to create modular routes, and middleware like 'isAuth' is implemented to verify user authentication token from cookies. Additional middleware for JSON parsing (express.json) and cookie parsing (cookie-parser) are added to the main Express application ('index.js'). CORS is configured to allow requests from the React frontend.

The React frontend is set up using Vite. Tailwind CSS is integrated for styling by installing the necessary packages and configuring the 'vite.config.js' and 'index.css' files. React Router DOM is installed for client-side routing, and Axios is added for making HTTP requests to the backend. Essential pages like 'Signup.jsx' and 'Signin.jsx' are created to handle user authentication.

A User Context API is implemented to manage and provide user data, including the server URL, throughout the React application. This ensures that user details, such as the current logged-in user, are accessible globally. Route protection is set up in 'App.jsx' to ensure users are redirected to the appropriate pages (e.g., 'Customize' or 'Home') based on their authentication status and whether their assistant is already configured.

Cloudinary is integrated for image storage. A 'cloudinary.js' file is created in the backend's 'config' folder to handle image uploads. API credentials for Cloudinary are stored as environment variables. A Multer middleware is also implemented to temporarily store uploaded images on the server before sending them to Cloudinary and then deleting the local copy to save storage.

Two customization pages ('Customize.jsx' and 'Customize2.jsx') are built for the virtual assistant. 'Customize.jsx' allows users to select a default assistant image or upload their own, providing visual and functional feedback on selection. 'Customize2.jsx' then prompts the user to name their assistant. These inputs are used to update the user's profile in the backend, linking the chosen image and name to their account.

The Google Gemini API is integrated into the backend to power the assistant's natural language processing. A 'gemini.js' file handles API requests, sending user prompts to Gemini and processing its responses. A detailed prompt is crafted to instruct Gemini on how to behave as a virtual assistant, including specific task types (e.g., general query, Google search, YouTube search, time, date, weather, calculator, social media commands) and the desired JSON response structure.

Web Speech API is used for speech recognition (converting voice to text) and speech synthesis (converting text to voice). The system is configured to continuously listen for the assistant's name, activating only when addressed. A `speak` function is created to enable the assistant to vocalize its responses. The home page UI is designed to dynamically display user inputs and assistant responses, including a GIF that animates when the assistant is speaking.

User commands are stored in the user's history in the MongoDB database. A responsive sidebar/hamburger menu is implemented for smaller screens to house the 'Customize' and 'Logout' buttons, along with the interaction history. The assistant's voice is customized to provide responses in Indian Hindi, complete with a female voice accent, enhancing the user experience.

The final virtual assistant is thoroughly tested for functionality across various commands, including system queries (time, date), web searches (Google, YouTube), and application launches (Calculator, Instagram, Facebook). The project is then prepared for deployment by making necessary changes for GitHub (e.g., '.gitignore' for '.env' files, '.gitkeep' for empty folders). Finally, both the backend (web service) and frontend (static site) are deployed to Render, ensuring proper configuration of environment variables and CORS policies.