Progress Highlights
Project Research
No progress made on project research since Week 6 is dedicated solely for the development of the labelling system.
Project Development
NOTE: Because the deadline of the project is Week 7, a lot of the labelling system's development has been crunched within 1 week.
Monday (12/4/2021):
Made progress on essential parts Tweets Labelling module for Expert (Label tweets, calculate result automation, display labelled tweets at UI)
- Fixed custom Account model bugs for AccountManager, UserCreationForm and UserChangeForm (revamped AccountManager, changed UserCreationForm to subclass UserCreationForm instead of form.modelForm)
- Fixed some UI responsiveness issues when viewed in portrait and landscape mode (removed footer in mobile, repositioned log out link at sidebar, added overflow-hidden at main, removed whitespace-nowrap for table rows)
- Updated generate-expert.html template (centered header)
- Created 2 new models for Tweets Labelling module (Label and ExpertSet - bridge entity between Set and Expert/Account)
- Updated Set model with new no_of_tweets field and IntegerField validators
- Updated custom Account model fields with verbose name and help texts
- Added URL and FBV for test in administrator and expert app
- Updated FBV for generate_expert (update the assigned expert no in tweet set and create new ExpertSet object upon expert account creation)
- Updated FBV for upload_set (added Result set column and updated Set.objects.create command with new fields)
- Added label.json model seed for Label model
- Updated FBV for expert_home (read tweets set of expert, store all labelled tweets, write labelled tweets to tweet set, auto-calculate result for all tweets, upload updated tweets set in local and cloud, display expert labels to front-end)
- Updated template logic for expert-home.html to dynamically display and select the label options
- Renamed tests.html to test.html and added codes from Faidz
- Commented nav__logout CSS in base.css
Wednesday (14/4/2021): Completed all basic functionalities for several modules (Tweets Labelling, Expert Management, Tweets Set Management)
Models:
- Created 2 new fields (set_progress and set_status) for Set model (Show Set labelling progress)
- Created 3 new fields (expert_progress, expert_status and expert_page) for ExpertSet model (Show Expert labelling progress, Divide tweets into pages)
- Remove assigned_set field in custom Account model (redundant due to existence of ExpertSet model)
- updated account/admin.py for newly-added fields
Expert Management:
- Added new columns inside Expert Management table (Labelling Status, Labelling Progress)
- Combined Generate Expert template into newly created Experts Management template
- Removed Student ID form input field occurrence from template and views
- Changed return redirect URL for Generate Expert to Expert Management
- Added Tailwind-Alpine dynamic modal for Delete Expert and Generate Expert Report
- Configured FBV and URL for Generate Expert Report (using csv library)
- Configured FBV and URL for Delete Expert
- Amended Generate Expert FBV for changes in custom Account model and ExpertSet model
Tweets Set Management:
- Combined Upload Tweets Set template into newly created Tweets Set Management template
- Changed return redirect URL for Upload Tweets Set to Tweets Set Management
- Added Tailwind table for Tweets Set Management
- Added Download Tweets Set function for each set in Tweets Set Management table using hyperlink
- Added Tailwind-Alpine dynamic modal for Delete Set and Generate Set Report
- Configured FBV and URL for Generate Set Report (using csv library)
- Configured FBV and URL for Delete Set
- Removed 'N/A' set instance and its .exclude() occurrences in views.py
Tweets Labelling:
- Added functionality to read/write labelled tweets set into CSV file
- Added functionality to automatically calculate result for all labelled tweets
- Fixed bug of tweets labelling result calculation by replacing sum() and mean() Pandas function with mode()
- Added try-except KeyError block for cases where no mode can be detected (default to 0)
- Added upload updated tweets set functionality to both local (MEDIA_ROOT) and cloud (Firebase Cloud Storage)
- Added expert progress and status functionality (update status to 'Completed' if progress reaches 100)
- Added functionality to display labelled tweets of experts at template
- Fixed bug for displaying tweets result from CSV file by casting EXPERT_COL with float
- Added emojize functionality to convert emoji descriptions of tweets from CSV file into emojis for template display
- Added success message for tweets labelling
- Added functionality of automatic account deactivation for Experts after labelling completion (set is_active field to False)
- Added functionality to render assigned tweets set of expert onto template in table form
- Utilised Django Paginator to divide tweets into multiple pages
- Amended Expert Home template with Django Paginator tags for pagination using Tailwind
- Renamed tests.html to expert-test.html
General:
- Added STATICFILES_DIRS directory in settings.py to cater for non-app static files
- Temporarily set ALLOWED_HOSTS to all for ngrok usage
- Temporarily added static URL in project urls.py and MEDIA_URL in settings.py to allow for file download from MEDIA
- Moved base static files and template into project static directory
- Renamed test.html to admin_test.html (including its URL and FBV)
- Created new draft directory in administrator and expert app for template drafts
- Added functionality to close sidebar by clicking anywhere at body using JavaScript
- Fixed margins/paddings of templates
- Moved log out button to top instead of bottom of sidebar
- Fixed Tailwind-Alpine dynamic modals glitch by adding x-cloak and x-data classes in body tag
- Removed social media links in footer
- Commented several hyperlinks from sidebar for both Expert and Administrator
- Updated requirements.txt for emoji library
Thursday (15/4/2021):
Implemented changes to system functionality and UI/UX based on comments from supervisor
Account:
- Added allowed_users decorator in decorators.py
- Temporarily commented admin_home redirect in unauthenticated_user decorator (Administrator Dashboard not ready yet)
- Added unavailable_page URL and FBV in views.py
- Updated is_staff and is_superuser verbose names for custom Account model
Expert Management:
- Added search bar and labelling status dropdown filter at main table using JavaScript
- Created new admin.js file
- Added Tailwind-Alpine dynamic modal for Edit Expert
- Configured URL and FBV for Update Expert (update username, email, full name and password) -Added if-else validation for generate_expert_report and generate_expert_report for no records case
Tweets Set Management:
- Added search bar and set status dropdown filter at main table using JavaScript
- Added Download Sets functionality with validation for no sets (configured URL, FBV and template; download all sets in .zip file)
Tweets Labelling:
- Added auto-redirect to Expert last-saved tweets progress page using JavaScript
- Added temporary Note alert reminding experts to ensure all tweets in current page have been labelled before proceeding to next page
- Fixed bug for success message of labelling submission by changing redirect(expert_home) to redirect(reverse(expert_home))
- Changed NO_OF_TWEETS_PER_PAGE from 4 to 20
- Changed Tweet label value from [-1, 0, 1] to [1, 2, 3] and Neutral' Tweet label to 'Neither'
- Alter field label_val on label (changed MinValue and MaxValue validators)
- Add 'Save' button at bottom of tweets labelling table
- Add English welcome message and instruction to Tweets labelling UI
General:
- Appended allowed_users decorators to all FBVs
- Added unavailable_page.html template and CSS for allowed_users decorator
- Temporarily commented admin_home / Administrator Dashboard in sidebar
- Added new separate logout template for redirect upon logout (configured logout and expert_home FBV)
- Temporarily hide Forgot Password, Remember Me, Create an Account from login template
- Changed username and password format for Generate Expert Account to student ID and student ID + 'xyz
- Added new Username and Full Name headers in Expert Management table
- Added new First Name and Last Name input fields inside Generate Expert Account form (updated generate_expert FBV)
- Changed all {{message}} occurrences into {{message|linebreaksbr}} to fix newline issue
- Added new Fitweet-Tailwind modal for Generate Expert Report with column checklists
- Updated generate_expert_report FBV with dynamic CSV file generation
Friday (16/4/2021):
Added Generate Bulk Experts functionality for Expert Management
- Added 2 new URLs (generate_bulk_experts and download_bulk_csv) for Generate Bulk Experts functionality
- Configured generate_bulk_experts FBV for registering Expert accounts in bulk with necessary validation for null values, duplications and unavailable check using If-Else and Try-Except block
- Configured download_bulk_csv FBV to handle file download for newly-generated bulk Expert accounts
- Created new directory and file (static/file/bulk_expert_template) for Generate Bulk Experts functionality
- Added JavaScript function called reloadDownload() to handle generated_experts.csv file download and page refresh for displaying messages
- Added Warning message template in Experts Management page
- Moved all inline JavaScript codes in Tweets Labelling page to newly created expert.js file
- Added personal emails at Instructions of Tweets Labelling page for contact purposes
- Added unique=True constraint for set_name field in Set model
- Add missing JS function for sidebar logout
- Replaced cursor-not-allowed to pointer-events-none for pagination buttons
- Renamed Dashboard label to Tweets Labelling of sidebar
Saturday (17/4/2021):
Deployment of labelling system to Azure Virtual Machine (VM) using NGINX and Gunicorn
- Updated requirements.txt and settings.py for deployment
- Fixed conflict in Django versions by replacing BASE_DIR from Path to os.path.dirname() in settings.py
- Added STATIC_ROOT url in settings.py
- Used collectstatic command to collect all static files for production serve
- Fixed bugs involving AccountManager for createsuperuser and account creations
- Removed temporary static media root URL for development (media files hosting should be done by web server
- Updated MEDIA_URL
- Fixed bug for table filter due to change in columns and recollectstatic
- Deleted sample sets from media folder and amended expert-home template
- Bought domain name (fitweet.site) from Hostinger for RM5.30
- Changed ALLOWED_HOSTS to domain names
- Updated DATABASES in settings.py using PostgreSQL database instead of SQLite
Sunday (17/4/2021):
Configuration of custom domain name (fitweet.site)
- Changed nameservers (NS) in Hostinger setting to point to Azure NSs instead of Hostinger defaults
- Configured /etc/nginx/sites-enabled/fitweet_label to include server_name pointing to domain name
- Excluded migrations folder from gitignore and updated create_user method in AccountManager
- Changed Select All outline colour to gray
- Set DEBUG to False and SEO meta tags for login and base html
- Updated SEO Twitter meta tags
- Removed footer hyperlink
- Finally deployed the labelling system! Fitweet
What I'm stuck at?
- Filtering out 20 Tweets data sets of 500 tweets each for labelling
- Preparing for usage of labelling system by 60+ Experts starting Friday
How will next week be?
- Official commencement of Tweets data set labelling phase for 2 weeks
- Preparation for FYP monitoring session in Week 8
Lessons Learnt
Sometimes being too advanced in terms of the features you implemented might not be the best approach or solution to the problem statement. For instance, I initially implemented a random username and password generator for Experts account using Faker library. However, my supervisor argued that this will not bode well with the target users who have to memorise these random credentials. We had a bit of a heated argument over this but eventually I conceded after acknowledging the fact that the Experts who volunteered to label these datasets were not being paid and therefore we should at the very least try to make the process easier for them.
In relation to the point above, do not be afraid to try to defend your hard work. My supervisor has advised me once that in order to survive our upcoming viva at the end of the semester, we must be able to defend and justify the reasons as to why we do things a certain way. During the meeting when I presented the completed labelling system, my supervisor questioned a lot of the design decisions I made. Instead of acknowledging them immediately, I first attempt to provide the reasoning and justification.
Human Computer Interaction (HCI) is a crucial factor in UI/UX development. Despite putting a lot of emphasis on the UI development (especially in terms of responsiveness), there were some aspects of HCI that I missed out and was pointed out by my supervisor. For example, being able to use keyboard keys (tabs, up and down arrow) for navigating between table rows and select dropdowns will significantly ease the labelling work for Experts when using PC.