Technical SEO

12 min read

Updated 3 days ago

SEO Technical

Overview

The SEO Technical module provides comprehensive management tools for technical SEO aspects including robots.txt configuration, XML sitemap management, URL redirects, canonical URLs, and structured data implementation. These technical elements are crucial for proper search engine crawling, indexing, and understanding of your website's content structure.

Accessing SEO Technical

Navigation Path: Dashboard > SEO System > Technical SEO
Required Permission: blog.view (viewing), blog.edit (editing)
URL: /admin/seo/technical

Key Features

Robots.txt Management: Control search engine crawling behavior
XML Sitemap Integration: Automated sitemap generation and management
URL Redirect Management: Handle 301/302 redirects for changed URLs
Canonical URL Configuration: Prevent duplicate content issues
Structured Data Management: Schema.org markup implementation
Technical SEO Auditing: Identify and resolve technical issues
Redirect Testing: Validate redirect functionality
File Status Monitoring: Track technical file status and health

Main Interface

Page Header

The technical SEO page provides:

Page Title: "Technical SEO" with configuration subtitle
Overview Dashboard: Key technical metrics at a glance
Quick Actions: Direct access to management tools

Technical Statistics Dashboard

Four key metrics provide technical SEO status overview:

Robots.txt Status

Display: Active/Missing status with file size
Color Coding: Green for active, red for missing
File Information: Size in bytes and last modified date
Management Link: Direct access to robots.txt editor

Sitemap Status

Display: Generated/Pending status with URL count
Color Coding: Green for generated, yellow for pending
URL Count: Number of URLs included in sitemap
Management Link: Access to sitemap management interface

Redirects Count

Display: Total redirects with active count
Statistics: Total and active redirect counts
Management Link: Access to redirect management interface
Recent Activity: Count of redirects created in last 30 days

Canonical URLs

Display: Count of posts with canonical URLs
Missing Count: Posts without canonical URL configuration
Coverage: Percentage of posts with proper canonical setup
Optimization Opportunity: Identify posts needing canonical URLs

Robots.txt Management

Robots.txt Overview

Control search engine crawler access and behavior:

Purpose and Function

Crawler Control: Specify which parts of site crawlers can access
Crawl Budget Optimization: Direct crawlers to important content
Security: Prevent crawling of sensitive or private areas
Performance: Reduce server load from unnecessary crawling

File Status Monitoring

Existence Check: Verify robots.txt file exists in public directory
File Size: Monitor file size for completeness
Last Modified: Track when file was last updated
Accessibility: Ensure file is publicly accessible at /robots.txt

Robots.txt Editor

Direct editing interface for robots.txt configuration:

Editor Features

Syntax Highlighting: Visual formatting for better readability
Validation: Real-time validation of robots.txt syntax
Backup: Automatic backup before changes
Preview: Preview changes before applying

Common Directives

User-agent: *
Disallow: /admin/
Disallow: /private/
Allow: /public/

User-agent: Googlebot
Crawl-delay: 1

Sitemap: https://yoursite.com/sitemap.xml

Best Practices

Allow Important Content: Ensure important pages are crawlable
Block Sensitive Areas: Prevent crawling of admin and private sections
Include Sitemap: Reference sitemap location for crawlers
Test Changes: Validate robots.txt using Google Search Console

Robots.txt Templates

Pre-configured templates for common scenarios:

Standard Blog Configuration

User-agent: *
Disallow: /admin/
Disallow: /login/
Disallow: /register/
Allow: /

Sitemap: https://yoursite.com/sitemap.xml

Restrictive Configuration

User-agent: *
Disallow: /

User-agent: Googlebot
Allow: /
Disallow: /admin/
Disallow: /private/

XML Sitemap Management

Sitemap Integration

Automated sitemap generation and management:

Sitemap Features

Automatic Generation: Sitemaps generated from published content
Multi-format Support: XML sitemaps for search engines
Real-time Updates: Sitemaps update when content changes
Compression: Gzip compression for large sitemaps

Sitemap Statistics

URL Count: Total URLs included in sitemap
File Size: Sitemap file size and compression ratio
Last Generated: Timestamp of last sitemap generation
Validation Status: XML validation and error checking

Sitemap Configuration

Control what content is included in sitemaps:

Content Inclusion Rules

Published Posts: All published blog posts included by default
Static Pages: Important static pages included
Category Pages: Category listing pages
Tag Pages: Tag archive pages (optional)

Priority and Frequency Settings

Homepage: Priority 1.0, daily updates
Blog Posts: Priority 0.8, weekly updates
Category Pages: Priority 0.6, weekly updates
Static Pages: Priority 0.5, monthly updates

Sitemap Submission

Submit sitemaps to search engines:

Search Engine Submission

Google Search Console: Submit sitemap URL
Bing Webmaster Tools: Register sitemap
Robots.txt Reference: Include sitemap URL in robots.txt
Automatic Ping: Notify search engines of updates

URL Redirect Management

Redirect Overview

Manage URL redirects for changed or deleted content:

Redirect Types

301 Permanent: Content permanently moved to new location
302 Temporary: Content temporarily moved
410 Gone: Content permanently deleted
Custom Status: Other HTTP status codes as needed

Redirect Statistics

Total Redirects: Count of all configured redirects
Active Redirects: Currently functioning redirects
Inactive Redirects: Disabled or problematic redirects
Recent Additions: Redirects created in last 30 days

Redirect Management Interface

Comprehensive redirect management tools:

Redirect List View

Source URL: Original URL being redirected
Target URL: Destination URL for redirect
Status Code: HTTP status code (301, 302, etc.)
Created Date: When redirect was established
Hit Count: Number of times redirect was used

Redirect Actions

Create Redirect: Add new URL redirect
Edit Redirect: Modify existing redirect rules
Delete Redirect: Remove unnecessary redirects
Test Redirect: Validate redirect functionality
Bulk Operations: Manage multiple redirects simultaneously

Redirect Testing

Validate redirect functionality and performance:

Testing Features

URL Testing: Test specific URLs for redirect behavior
Status Validation: Verify correct HTTP status codes
Chain Detection: Identify redirect chains and loops
Performance Testing: Measure redirect response times

Test Results

{
  "old_url": "/old-page",
  "new_url": "/new-page",
  "status_code": 301,
  "response_time": "45ms",
  "redirect_chain": false
}

Canonical URL Management

Canonical URL Overview

Prevent duplicate content issues with proper canonical URL implementation:

Purpose and Benefits

Duplicate Content Prevention: Specify preferred URL version
Link Equity Consolidation: Concentrate ranking signals
Crawl Budget Optimization: Reduce wasted crawler resources
User Experience: Ensure consistent URL structure

Canonical URL Statistics

Posts with Canonical URLs: Count of properly configured posts
Missing Canonical URLs: Posts needing canonical configuration
Self-referencing Canonicals: Posts with self-canonical URLs
External Canonicals: Posts referencing external canonical URLs

Canonical URL Configuration

Set up canonical URLs for content:

Automatic Canonical URLs

Self-referencing: Posts automatically reference themselves
Clean URLs: Remove query parameters and fragments
HTTPS Preference: Prefer HTTPS over HTTP versions
Trailing Slash Consistency: Maintain consistent URL format

Manual Canonical Configuration

Custom Canonical: Specify custom canonical URL
Cross-domain Canonical: Reference external authoritative version
Parameter Handling: Handle URL parameters appropriately
Mobile Canonical: Separate mobile canonical configuration

Canonical URL Validation

Ensure proper canonical URL implementation:

Validation Checks

URL Accessibility: Verify canonical URLs are accessible
Self-reference Validation: Check for proper self-referencing
Chain Detection: Identify canonical URL chains
Consistency Checks: Ensure consistent canonical implementation

Structured Data Management

Schema.org Implementation

Implement structured data markup for better search understanding:

Structured Data Types

Article Schema: Blog post structured data
Organization Schema: Business information markup
Website Schema: Site-wide structured data
Breadcrumb Schema: Navigation breadcrumb markup

Structured Data Statistics

Posts with Schema: Count of posts with structured data
Posts without Schema: Posts missing structured data markup
Schema Types: Distribution of different schema types
Validation Status: Schema markup validation results

Schema Configuration

Configure structured data for different content types:

Article Schema Configuration

{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "Post Title",
  "author": {
    "@type": "Person",
    "name": "Author Name"
  },
  "datePublished": "2023-01-01",
  "dateModified": "2023-01-02",
  "image": "https://example.com/image.jpg"
}

Organization Schema

{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Your Organization",
  "url": "https://yoursite.com",
  "logo": "https://yoursite.com/logo.png",
  "sameAs": [
    "https://facebook.com/yourpage",
    "https://twitter.com/yourhandle"
  ]
}

Schema Validation

Validate structured data implementation:

Validation Tools

Google Rich Results Test: Validate schema markup
Schema.org Validator: Check schema compliance
Internal Validation: Built-in schema validation
Error Reporting: Identify and report schema errors

Technical SEO Auditing

Automated Auditing

Identify technical SEO issues automatically:

Audit Categories

Crawlability Issues: Problems with crawler access
Indexability Problems: Issues preventing proper indexing
URL Structure: Problems with URL configuration
Meta Tag Issues: Missing or problematic meta tags

Audit Results

Issue Severity: Critical, warning, or informational
Affected URLs: Specific URLs with issues
Fix Recommendations: Specific steps to resolve issues
Priority Scoring: Issues ranked by impact and effort

Manual Auditing Tools

Tools for manual technical SEO analysis:

URL Analysis

Redirect Chain Analysis: Identify redirect chains and loops
Canonical URL Validation: Check canonical implementation
Meta Tag Analysis: Review meta tag implementation
Schema Markup Validation: Verify structured data

Site-wide Analysis

Robots.txt Validation: Check robots.txt syntax and rules
Sitemap Analysis: Validate sitemap structure and content
Internal Link Analysis: Review internal linking structure
Page Speed Analysis: Technical performance metrics

API Endpoints

Redirect Testing API

Test redirect functionality programmatically:

POST /admin/seo/technical/test-redirect

Request Body

{
  "old_url": "/old-page"
}

Response Format

{
  "success": true,
  "redirect": {
    "old_url": "/old-page",
    "new_url": "/new-page",
    "status_code": 301
  },
  "status": "active"
}

Robots.txt Management API

Manage robots.txt content programmatically:

POST /admin/seo/technical/robots

Request Body

{
  "content": "User-agent: *\nDisallow: /admin/\nSitemap: https://site.com/sitemap.xml"
}

Troubleshooting

Common Issues

Robots.txt Not Working

Symptoms: Crawlers ignoring robots.txt directives
Solutions:

Verify robots.txt is accessible at /robots.txt
Check file permissions and server configuration
Validate robots.txt syntax using Google Search Console
Ensure no caching issues preventing updates

Sitemap Not Updating

Symptoms: Sitemap shows old content or missing URLs
Solutions:

Check sitemap generation cron jobs or triggers
Verify database contains published content
Clear sitemap cache if caching is enabled
Check file permissions for sitemap directory

Redirects Not Working

Symptoms: Redirects returning 404 or not redirecting
Solutions:

Verify redirect rules are properly configured
Check web server configuration (Apache/Nginx)
Test redirects using redirect testing tool
Ensure no conflicting redirect rules

Canonical URLs Not Recognized

Symptoms: Search engines not recognizing canonical URLs
Solutions:

Verify canonical URLs are properly formatted
Check that canonical URLs are accessible
Ensure canonical URLs are in HTML head section
Validate canonical implementation using SEO tools

Performance Issues

Large Sitemap Files

Problem: Sitemap files too large for efficient processing
Solutions:

Implement sitemap index files for large sites
Split sitemaps by content type or date
Use compression (gzip) for large sitemaps
Implement pagination for sitemap generation

Redirect Performance

Problem: Slow redirect processing affecting site speed
Solutions:

Optimize redirect rules for efficiency
Use server-level redirects when possible
Minimize redirect chains
Cache redirect rules for better performance

Best Practices

Robots.txt Best Practices

Configuration Guidelines

Allow Important Content: Ensure crawlers can access important pages
Block Sensitive Areas: Prevent crawling of admin and private sections
Include Sitemap Reference: Help crawlers find your sitemap
Test Regularly: Validate robots.txt using webmaster tools

Common Mistakes to Avoid

Blocking CSS/JS: Don't block resources needed for rendering
Overly Restrictive: Don't block content you want indexed
Syntax Errors: Validate syntax to ensure proper interpretation
Missing Sitemap: Always include sitemap reference

Redirect Best Practices

Redirect Strategy

Use 301 for Permanent: Use 301 redirects for permanently moved content
Minimize Chains: Avoid redirect chains longer than 3 hops
Update Internal Links: Update internal links instead of relying on redirects
Monitor Performance: Track redirect usage and performance